.png){kind=logo}
Troubleshooting Missing Leads After Importing from Apify to Clay
Hi, I imported 7.4K leads from Apify but only 6.6K showed in Clay, tried twice it's the same. How can I trouble shoot?
When importing leads from Apify to Clay, it's possible that not all leads appear due to limitations in data retrieval. While this specific issue isn't directly related to Apify, a similar situation can occur with other data sources. Here's what might be happening: 1. Data source limitations: Some platforms limit the number of leads that can be retrieved, which could explain why not all 7.4K leads are showing up in Clay. 2. Duplicate handling: Clay may be filtering out duplicate leads, which could reduce the total number of imported leads. 3. Data processing: There might be issues during the data transfer or processing stage.To troubleshoot this issue, you could try splitting your Apify data into smaller batches and importing them separately. This approach might help bypass any potential limitations.Could you provide more details about the Apify actor you're using and the specific configuration you've set up for the import? This information would help us better understand and address the issue. Source: Apify integration overview | Documentation | Clay University
Andi D.··how can I stop Clay from auto-dedup during import?
To stop automatic deduplication in Clay, you can turn off the auto-dedupe feature by: 1. Clicking the Auto-dedupe icon at the bottom right of your table 2. Toggle off "Enable automatic deduplication" 3. Click Save Changes Source: Auto-update and Auto-dedupe Table
Looks like you're still looking for help. We're escalating this over to our support team and someone should be in touch soon. In the meantime, feel free to include any other information that may be helpful in solving your issue!
Stefano··Hi! Hi, thanks for reaching out. To prevent duplicates from being removed from your table, you'll need to pause our "Auto-dedupe" function. Simply select the button at the bottom-right, as shown below: https://downloads.intercomcdn.com/i/o/w28k1kwz/1277865747/9b4d829761f4960c1bc6c9af2434/CleanShot%2B2024-12-05%2Bat%2B09_37_24-402x.png?expires=1736152200&signature=1a1290abb4fadfbaf6358d4f5de5afbd9a528c8bf92fbc9fe9df225ec8ac24c5&req=dSIgEcF4mIZbXvMW1HO4zZ9YkbL%2BhrG3Yl%2BMT7Ew88JkzA%3D%3D%0A From the menu that appears, toggle off the"Enable automatic deduplication" button. https://downloads.intercomcdn.com/i/o/w28k1kwz/1322555710/f610796a599a009e67daee50fe5b/CleanShot+2025-01-06+at+08_59_57%402x.png?expires=1736152200&signature=1b414a295e1f7eda23c8c833911b9257e6227904545854aad8b70645c98fcd54&req=dSMlFMx7mIZeWfMW1HO4zRaugNaY85Zl9zby%2BBCEsc5OzbxGL1rNyiQ3rePL%0A%2B4Xj%0A Next, click "Save changes". Let me know if this works!
- Andi D.··
Thanks Stefano! That part I got it. But still need help trouble shooting Apify--Clay import. I tried to import 7.5K ish records and only 6.6K would show in Clay. My initial guess was that 900 were duplicates and got auto-removed. But tried again after turning off auto-dedup and it's the same thing. Any idea on how to work around it?
- Stefano··
Do you mind sending the link to the table so we can take a look? https://downloads.intercomcdn.com/i/o/1171717996/ac7729f84b426e1d0358af02/image.png?expires=1736154000&signature=cb01de495d5abbc59896ce8bd8db1bd4fd80748968e5fa89f2305f5c36b2d179&req=dSEgF85%2FmohWX%2FMW1HO4zYTywS39A5MW2BKsam%2FhVPvGpg%3D%3D%0A
- Andi D.··
Absolutely. Here's the link: https://app.clay.com/workspaces/449716/workbooks/wb_GAzMXGodBkzz/tables/t_ke2QM3g63DiM/views/gv_NYB8tMBe9Awk I've cleaned out duplicates after the initial import so there's only 5.4K left
- Andi D.··
I've pulled leads from Apify twice so you will see 2 sources and a total of 13K ish initial leads being imported but 50% are same leads just imported twice
- Andi D.··
I've deleted the Apify actor run but it had 7474 records upon completion. I think Clay only captured around 6.5K
- Stefano··
Hi, I’m currently reviewing the table. One alternative we could try is downloading a CSV file from Apify that contains all the relevant information and then uploading it to Clay. Would this approach work for you? I’m a bit confused as to why only 900 records are being downloaded, and since I don’t have visibility into the original Apify list, it’s difficult for me to provide a definitive solution at this stage — I can only speculate. Please let me know if the CSV method would be okay, and I’ll be happy to assist further.
- Andi D.··
Tysm Stefano, so the issue with CSV is it contains wayyyy too many columns. Is there a way to upload it via Json? I tried to upload it to Airtable first and then have Clay pull from Airtable, but I'm also having difficulty with Airtable import right now LOL
- Stefano··
Hey, sorry for the delay! I’m still here, just working on figuring out the best course of action. I’ll get back to you soon with a response — I just want to make sure I’m providing the correct information. Thanks so much for your patience!
- Andi D.··
No problem, Stefano! I still have the CSV (7474 records) downloaded from Apify. Lmk if it’s needed
- Stefano··
Hi, apologies for the question, but do you definitely need more than 72 columns from the CSV file? If not, I’d recommend removing any unnecessary columns before uploading it. I suggest keeping the Apify ID, as it will be helpful later for re-uploading the information to the correct place on Apify. For instance, I’ve noticed that there are a lot of company-related details that I don’t think you’ll need. However, I do recommend keeping the person’s full name, as well as the LinkedIn URLs for both the person and their company, since these can be useful for finding emails or additional information.
- Stefano··
For example, as you can see here: https://downloads.intercomcdn.com/i/o/w28k1kwz/1322651096/32296fc2d93538af7b178c805d86/CleanShot+2025-01-06+at+10_34_54%402x.png?expires=1736157600&signature=1acc5ddcab3d260f2031656ec431ed25f475722da2a2762f2176be6fc01ffe7f&req=dSMlFM97nIFWX%2FMW1HO4zRJrQrJNOaoI1Z4GAw5MokmN3iNCztLmEQa6fRYC%0A77D2%0A A lot of the columns are not really useful as they contain no information you would need.
- Stefano··
I am sorry if this is a long-winded way, I am just very confused as to why 900 records would not be importing and all the other ones would, if it is not due to autodedupe.
- Andi D.··
Not a problem, thanks a lot Stefano! For this upload I can do that no problem but hopefully you guys can find out what went wrong for future references!
- Stefano··
Hi! I’ve reached out to other members of the support team to help identify the cause of the issue. I’ll be sure to provide you with an update as soon as I have more information on why this has occurred. This should be a one-off situation, and I truly appreciate your patience and understanding! :)
Hi, do you have any experience with importing json files to Clay?
Bo (.··Hey Andideg! Thanks for reaching out! Typically, you can only import CSV files. If the JSON is inside the CSV as a unique row, we can format this inside another table by using the "write to other table as individual rows" feature. For example, you can add your JSON data inside the Clay table, use a formula to turn this into an object [], add the "write to other table" action, click on the ⚙️ inside the list item, and use JSON.parse(/linkyourcolumncontainingjson). This will allow you to thread your JSON object as a list and send it to another table as individual columns. Let me know if you need further assistance or clarification! 🚀
Thanks Bo, this makes sense!