Need Help Troubleshooting Apify to Clay Data Import Issues

Hey, I’ve built a few working scrapers, but when the data gets imported into Clay daily, it reimports the entire dataset, not just the new rows. Not sure if the problem is with:

the Apify actor
the scheduled task in Apify
or the integration setup between Apify and Clay.

Would really appreciate some help troubleshooting this!

8 comments

Clay
APP
·
·
When using Apify with Clay, to import only new data rather than the entire dataset each time, make sure you don't specify a particular run as an input. Instead: 1. In the Source Modal, select Apify (under "other") 2. Leave the "Get Data from Specific Run" option unchecked 3. Don't specify a particular run ID This setup will automatically update your table with only new data from subsequent future runs of your actor, refreshing every 24 hours.
Sources: New at Clay | Clay.com Pull Data from Apify Actor with Apify
Elric L.
·
·
I thought I did, so I will try again with a new table. or do I have way to update tis setting after creating my table?
Clay
APP
·
·
Yes, you can update your table's auto-update settings after creation. To modify how your Apify integration imports data: 1. Click on your table name in the top bar 2. Select "Enable auto-update" or "Disable auto-update" from the dropdown menu This controls whether Clay automatically processes new data coming in from your Apify integration.
Sources: Table management settings | Documentation | Clay University Salesforce | Documentation | Clay University FAQ: What are the default sync settings for CRM integrations?
Elric L.
·
·
Can I update the the APify integration? not the auto-update
Clay
APP
·
·
To update your Apify integration settings for an existing table: 1. Go to Settings > Connections in your Clay dashboard 2. Find and click on the Apify integration 3. Click the "..." next to your Apify account 4. Select "Edit" to modify your account details This allows you to update your API key or rename your account.
Sources: Apify integration | Documentation | Clay University Managing integration accounts | Documentation | Clay University
Elric L.
·
·
https://app.clay.com/workspaces/451915/workbooks/wb_0sxatjk4wHTzg9Rqipk/tables/t_0sydtqigWKUsyRSPrQi/views/gv_0sydtqic76Bm3VwGmJZ
Elric L.
·
·
just checked, I had toggled off “Get Data from Specific Run.”
Bo (.
·
·
Hey, glad to hear that toggling “Get Data from Specific Run” helped. Another option you might consider is using webhooks — that way, the data can be pushed directly into Clay only when new runs are completed. Let me know if you’d like help setting that up.

Need Help Troubleshooting Apify to Clay Data Import Issues | Clay