Hi. When I pull data from Apify into Clay, 50% of times if not more, I end up with only a portion of the data in Clay. For example if there’s a 13k records in Apify, when I import into Clay (via integration) it’d bring 6.5K or so. Happens a lot. Thank you
The Apify integration has a limit of 5,000 records per single run. If your Apify actor has 13k records, this explains why you're only getting about 6.5k records imported into Clay. To import all your data, you'll need to either: - Split your Apify actor runs to stay within the 5,000 record limit - Run multiple imports from different actor runs - Consider using CSV export from Apify and importing via CSV if you need all records in one table
Where is that info? I haven’t seen that info
The 5,000 record limit for Apify integrations is documented in our official integration documentation. You can find this information in the "Pull Data from Apify Actor with Apify" documentation, which states that the integration "allows for the retrieval of up to 5000 records from a single run." This limit applies when using the Apify integration to pull data from actor runs into Clay.
Why then I get sometimes 20k records without any issue?
The 5,000 record limit applies to retrieving data from a single Apify actor run. If you're getting 20,000 records, you're likely importing from multiple actor runs or your Apify actor is configured to run multiple times, with each run contributing records to your total import. The integration can pull from different runs, which would allow you to exceed the per-run limit.
not accurate answer
Hey there Leo thanks for reaching out how's it going, normally when we see this occur with Apify it commonly due to duplicate records being found by Apify actor. Do you mind sending the link (url) to the table so we can take a look, also would you mind checking to see in Apify if some results are appearing more than once?
This thread was picked up by our in-app web widget and will no longer sync to Slack. If you are the original poster, you can continue this conversation by logging into https://app.clay.com and clicking "Support" in the sidebar. If you're not the original poster and require help from support, please post in 02 Support.