Hi Clay support and community - got an issue. For context, I’ve been successfully using Clay and Apify for about a month now! Before I could let it run overnight and it would scrape data for 5000 leads plus without me having to touch it. But overtime it’s gradually gotten worse and worse to the point of now the Apify actor only scrapes data for just one lead before stopping. I’m in contact with Apify’s support team for the last half a week and we’re still struggling to solve this. They suggested I try increasing the memory within Apify. I went ahead and took their advice on adjusting the allocated memory for each run. I changed it from 4mb to 32mb (which is the maximum on my current paid plan with Apify). It was an improvement. Instead of it scraping only one before stopping, it scraped three before stopping. But obviously, that doesn’t come as much help if I have thousands of leads & websites to scrape. Something to note that I found strange, which made me think it could be a Clay issue rather than an Apify issue like I originally thought. I managed to set up a second Apify account with another email address (you get $5 free credits). I thought perhaps if I used a fresh Apify account I wouldn’t run into the same issue because it worked so well at the beginning with my first Apify account. But weirdly I did. Yet again, it was only able to scrape one or two and then couldn’t do anymore (approx $0.50 was spent before it stopped). Any ideas on how I can fix this problem? And whether it’s an Apify problem or Clay problem in your opinion? Thanks in advance 🙂
👋 Hey there! Our support team has got your message - we'll be back in touch within 24 hours (often sooner!). If you haven't already, please include the URL of your table in this thread so that we can help you as quickly as possible!
Hey, thanks for reaching out! Sorry you’re experiencing issues with Apify. It could be that you’re requesting too much data, which might be causing the issue. Reducing the size of your request could help. I’m happy to check your table setup and see if there are any incorrect settings that could be causing the problem. However, Apify’s support might be better equipped to assist you directly. Let me know if you’d like me to take a look! 😊
Thanks bro. How do I can reduce the size of my requests? I'm assuming this is in Apify?
Hey Jacques! Thanks for the reply. Currently, there isn't a way to reduce the size directly in Clay unfortunately. However, if an Apify actor's Curl has a line for limiting the number of results returned, you can reduce the limits in the body field (see screenshot below). I recommend checking with Apify support to see if there's a workaround on their side for the request size. Let me know if this helps. https://downloads.intercomcdn.com/i/o/1177488189/e2b000ff49a01e46fc3d49d4/CleanShot+2024-09-12+at+11_01_48%402x.png?expires=1726137000&signature=6642b8bed3cd3be36d0ccafecc001af8daafb6c57183ab2dda2c68c77c8b0cdf&req=dSEgEc12lYBXUPMW1HO4zTIlwMMKfzTr1jaUFtx%2FEXZiohsOccDXq4pemK2g%0A51EM%0A
We haven't heard back from you in a bit, so we're going to go ahead and close things out here - feel free to let us know if you still need something!
Hi Jacques S.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:
