Understanding Auto-Delete Timing for Large Tables in Clay

How long does "auto-delete" take before it starts deleting the old rows? I have a 31k table, and have been waiting ~20 mins. Is it based on the load the Clay servers are facing, is it a periodic cronjob? Can someone shed some light on this?

16 comments

Channeled
APP
·
·
Auto-delete is designed to automatically process incoming data, enrich it, and forward it to a designated destination before deleting the original entries from the table. The feature only works with tables where the source is webhooks and requires enabling "Automatic row deletion" in the table settings. You can also enable a "Success field check" to confirm that a column has run successfully before deleting any rows. The documentation doesn't specify the exact timing, server load impact, or scheduling details for when auto-delete begins processing rows. What type of data source are you using for your 31k table - is it webhook-based data or another source type?
Sources: Auto-delete in tables | Documentation | Clay University Auto-delete in tables | Documentation | Clay University Auto-delete in tables | Documentation | Clay University
Siddhant B.
·
·
https://app.clay.com/workspaces/39601/workbooks/wb_rYi2eVkahgPD/tables/t_URRUd5qsFqS4/views/gv_gWkPGs3vGetn
Bruno R.
·
·
Hi Siddanth, with passthrough tables, you can essentially have rows in two different statuses: those that meet the deletion criteria and those that don't. Those that meet the deletion criteria will always be deleted on a first-in-first-out basis, and we preserve the latest 1000. So if a new row comes in that meets the deletion criteria AND there are already 1000 of these rows in the table, we'll delete the oldest row that also meets the deletion criteria.
Let me know if this makes sense!
Siddhant B.
·
·
Right - I think I got that logic. There are 31k rows in total, some of which go back 2-2.5 months - but all updated within the last 8 days. Could that be preventing the deletion?
Bruno R.
·
·
In this case, you set up auto-delete to run without a specific deletion criteria:
(183 kB)
This results in rows only being deleted if all enrichments in a given row have ran successfully. I'd suggest setting up the deletion criteria to run on a specific success field instead here.
Siddhant B.
·
·
What an interesting revelation - thanks for sharing! I'll need to recraft my table a bit to work according to the above logic.
Bruno R.
·
·
Happy to share context here. Let us know if you run into any other issue here. Happy to help Siddhant!
Siddhant B.
·
·
Bruno R. - one of the columns was returning an error, but I rectified it yesterday, however auto-delete still doesn't work. The number of rows is 31791 : https://app.clay.com/workspaces/39601/workbooks/wb_rYi2eVkahgPD/tables/t_URRUd5qsFqS4/views/gv_gWkPGs3vGetn
Siddhant B.
·
·
Since our client has been stuck on this for a while now, we will delete old rows manually in order to start new data coming in again.
Bo (.
·
·
Hey, Sorry about this. Just to confirm—are you trying to keep the table at 15,000 rows? If so, would it be okay for me to run a few tests? I’d try toggling the auto-delete setting off and on again to see if that resets it properly.
Siddhant B.
·
·
Yes please! Thanks!
Siddhant B.
·
·
And yes - trying to keep it under 15k
Bo (.
·
·
Got it—sounds like there might be a bug here, but I’d love to test it with a successful run. Would it be okay if we use something like the Normalized Domain column as a success check to see if that triggers auto-delete properly? Also, just curious—have you seen the row count go above 31,791, or has it stayed locked there the whole time? That might help us narrow things down.
Siddhant B.
·
·
Yes - it would be fine. Please avoid re-triggering the final HTTP API call (which sends data into an external system). Sorry for delayed response. Appreciate you looking into this!
Siddhant B.
·
·
I think you did something and now the row count is down to 15k - thanks Bo (. Bo (.!
Gabe E.
·
·
Hi Siddhant, I'm glad my teammate was able to sort this out. Please let me know if you're still having issues here and how I can help!
Gabe
1

Understanding Auto-Delete Timing for Large Tables in Clay | Clay