Implementing Sequential Runs for Apify Tasks to Avoid Errors | Clay

Implementing Sequential Runs for Apify Tasks to Avoid Errors

I'm trying to run an apify task call from Clay.com. When I run one row manually, it works. But, when I run the column, i get an error. Is there a way (setting in Apify or edit to my code) to implement sequential or staggered runs if multiple Actors are currently running in parallel? Appreciate any help!

16 comments

Owen C.
·
·
Hey Sean, thanks for reaching out! Which Apify Actor are you running? also do you mind sending over the URL to this Clay table so we can take a closer look?
1
Sean M.
·
·
I'm running Contact Info Scraper. Sure, here it is: https://app.clay.com/shared-table/share_NJsxVU76eHVJ
Owen C.
·
·
Got it, do you minding sharing the URL to the Clay table where this error is occurring? This link appears to be the template used. Regardless, I believe you could use the maxConcurrency setting within Apify to control the maximum number of concurrent requests. By setting this to 1, you can enforce that only one request is made at at time which effectively makes this process sequential. In Apify, this would look like: maxConcurrency: 1,
Sean M.
·
·
Ok, here's the url: https://app.clay.com/workspaces/415777/tables/t_J8ZA76MCaBr7/views/gv_Dx6E4BYKz7qy
Sean M.
·
·
That makes sense. Would I add "maxConcurrency: 1," to the JSON?
Sean M.
·
·
Unfortunately, even w/ maxconcurrency:1 I'm still running into the memory limit error.
Owen C.
·
·
Got it, what does your JSON input look like right now? Apologies, it's sometimes challenging to troubleshoot without the full visibility into your side of Apify. I'd be curious if the following would work as JSON input: { "start_url": "https://www.example.com", "depth": 1, "maxConcurrency": 1 }
Sean M.
·
·
{ "considerChildFrames": false, "maxDepth": 2, "maxRequests": 1000, "maxRequestsPerStartUrl": 2, "sameDomain": true, "startUrls": [ { "url": "WEBSITE", "method": "GET" } ], "maxConcurrency": 1 }
Owen C.
·
·
Got it, as I'm checking this again it appears to be working in a proper manner. Most of these emails appear to be general business emails which I understand may have to be utilized for reaching out to businesses such as this. Which change did you make that allow you to get improved results? https://downloads.intercomcdn.com/i/o/w28k1kwz/1250750739/84a8827db3a8fc22f322e2b56e7f/CleanShot+2024-11-13+at+13_43_10.png?expires=1731525300&signature=3aa30feb359ea174edc59b5966014364ff25807006869f5d95a6fbb3c2f7d877&req=dSIiFs57nYZcUPMW1HO4zebfaKwx1J4XzRYRIUd692BYZ%2Fpab170p579h1ZO%0AirFV%0A
Sean M.
·
·
So, those are all rows I ran manually (one at a time). If I try to run multiple rows at once, it only runs 2 rowas, and gives errors for the rest of them...
Owen C.
·
·
Ah got it, thank you. Looking into this further
1
Owen C.
·
·
Hey Sean, after checking in with our team we've confirmed this is a limitation on Apify's end. Since Clay processes rows independently to Apify, you'll need to upgrade your Apify plan to handle concurrent requests. The current 8192MB memory limit on your Apify account is causing this error when multiple rows try to run simultaneously. You can check out Apify's pricing here: Apify Until you upgrade, you can: 1. Process fewer rows at a time 2. Split your data into smaller batches Hope this clears things up! Let me know if you have any other questions :)
1
Sean M.
·
·
Ah, okay, I really appreciate the time you took trying to help me find a solution! I appreciate it.
Owen C.
·
·
Of course! Let us know if you have any other questions or concerns :)
1
Channeled
APP
·
·
Hi Sean M.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:
Channeled
APP
·
·
Thank you so much for sharing your feedback Sean M.!