Speeding Up Airtable's Lookup Record Enrichment for Duplicates

Using Airtable's "Lookup Record" Enrichment to check for duplicates in our Airtable database. The enrichment is painfully slow - up to 5 minutes for 10 records. Wondering how I can troubleshoot that + potentially speed it up

14 comments

Joe H.
·
·
Clay
APP
·
·
To troubleshoot slow Airtable Lookup Record enrichments: Processing in smaller batches - Break your data into smaller batches of around 50 rows at a time instead of processing all at once. Check for stuck columns - If the lookup is stuck on "queued," click the column header and select Force Rerun Column to reset it. Force restart if needed - Click the column header, hover over the Run column field, and select Force to jump start the run. Performance considerations - Large tables and multiple simultaneous actions can cause latency issues. Try a hard refresh (Mac: ⌘+Shift+R, Windows: Ctrl+F5). Optimize queries - Ensure your search queries are specific and well-defined, and monitor your API quotas. Processing 10 records in 5 minutes suggests potential API rate limiting or performance bottlenecks that these troubleshooting steps should help resolve.
Clay
APP
·
·
Since you've added an attachment, I can see you're providing additional context about your slow Airtable Lookup Record enrichment issue. Based on the troubleshooting steps I mentioned earlier, here are the key actions to try: Force restart the enrichment: ** Click the column header of your lookup column ** Hover over "Run column field" ** Select "Force" to jump start the run Process in smaller batches: ** Break your data into batches of around 50 rows instead of processing all at once Performance optimization: ** Try a hard refresh (Mac: ⌘+Shift+R, Windows: Ctrl+F5) as large tables and simultaneous actions can cause latency issues Optimize your queries: ** Ensure search queries are specific and well-defined * Monitor API quotas and usage limits The 5-minute processing time for 10 records suggests API rate limiting or performance bottlenecks that these steps should resolve.
Joe H.
·
·
I can't run this in 50 row batches
Joe H.
·
·
https://app.clay.com/workspaces/192229/workbooks/wb_0szig8rrsU69YuTrz48/tables/t_0szig8zJK7xeTPn96ze/views/gv_0szig9049ffEHWVWBoz
Joe H.
·
·
I need this enrichment to run reliably so that I can lookup records in our Airtable CRM and prevent duplicate sales outreach
Joe H.
·
·
I really need someone who is experienced with Clay and Airtable
Clay
APP
·
·
Hi there! Thank you for your patience. After reviewing the logs, I can see that there is a timeout message because the Airtable database is taking longer than 100 seconds to respond to the lookup queries This could occur if your Airtable is extremely large. Is your Airtable large? One workaround is to use Clay's "Lookup Value or Formula". Please let me know if you have more questions!
(257 kB)
Joe H.
·
·
Ok, any guidance on formula creation?
Joe H.
·
·
I tried it in the syntax you presented as well as a few others. When using the formula the record lookup completes faster but does not correctly identify matching records
Joe H.
·
·
My Airtable base is not super large - about 4500 total records
Clay
APP
·
·
Happy to jump in for Natalie here. It sounds like the Airtable lookup is running but not actually identifying matches correctly. That’s usually a sign the formula isn’t quite evaluating the results the way we want it to. One solid way to check for a match is to use this in a formula: LBKS!!{{Airtable Lookup}}?.records?.[0]?.EmailLBKSThat gives you true if a match was found, and false if not. You can also use the foundMatches field that comes with the lookup: LBKS{{Airtable Lookup}}.foundMatches = falseLBKSAlso, something to watch out for - if your lookup value (like a domain or email) is blank, Airtable can sometimes return a “match” just because both sides are empty. To prevent that, you can wrap the lookup in a quick check like this: LBKSIF({Company Domain} != "", {{Airtable Lookup}}, "Skip")LBKSAnd just a quick reminder, it’s best to match on something like domain, email, or a unique ID instead of company name, which can be inconsistent. Formula errors are super common too, especially with column names that have spaces or dashes. Using the / column picker or wrapping the name in {curly braces} helps avoid that. If it helps, a nice setup that works well is: ** First, a column that checks if your lookup value (like domain) exists ** Then, the Airtable lookup itself * Then a formula column that decides what to do next, based on whether there was a match Let me know if that helps
Joe H.
·
·
That worked thanks!
Clay
APP
·
·
Great! Anything else I can help you with in this workflow?

Speeding Up Airtable's Lookup Record Enrichment for Duplicates | Clay