Hi guys, quick question regarding GPT enrichment efficiency. I’m currently running a list of 6,700 rows. To avoid hitting rate limits or slowing down the system, I’ve been manually running batches of 100–200 rows, waiting for completion, and then starting the next batch. Is there a way to automate this sequential batching within Clay? Ideally, I’d like to trigger the next 200 rows automatically once the previous set finishes, rather than doing it manually. Any "auto-throttle" or sequencing settings I should know about? Thanks!
Clay now has Scheduling that can automate column runs on a daily, weekly, or monthly basis, but it doesn't handle the specific sequential batching you're looking for. [1]
For sequential workflows, you can use "Only run if" conditions to create dependencies between columns - like having Column 2 only run after Column 1 completes. However, this doesn't solve automatic batching of the same enrichment.
Before we explore other options, I'd recommend trying Sculptor (our built-in AI tool in the top-right corner). It's free and might help you set up a more automated workflow for your specific batching needs. Can you give Sculptor a try and let me know if it helps with your sequential batching setup?
Okay, I'll try Sculptor and get back to you if I have further questions.
Thank you for letting me know. Feel free to reach out whenever you have more questions or need further assistance.
If you still need help with setting up automated sequential batching or using Sculptor, I’m here to assist. Would you like to share more about what you’re trying to achieve or any challenges you’re facing with Sculptor?
You could run each batch in separate columns on the condition of the previous one completing, then write them all to a joined column after every batch concludes
That's exactly right! You can create separate columns for each batch and use conditional logic to sequence them.
Here's how to set it up:
1. Create multiple GPT columns (one for each batch of 200 rows)
2. Use "Only run if" conditions to create dependencies - Column 2 only runs after Column 1 completes, Column 3 after Column 2, etc.
3. Use Clay.getCellStatus() formula for more precise control: Clay.getCellStatus(#{{field_id}}) - this checks if the previous column has actually completed
4. Create a final formula column to combine all batch results into one unified output
This approach gives you the sequential batching you want while avoiding rate limits. Each column will automatically wait for the previous one to finish before starting. [1]
The "Only run if" conditions ensure proper sequencing, and you can even add delay columns between batches if you need additional spacing.
Did that answer your question?
If you need any more help with setting up your sequential batching or using Sculptor, I’m here to assist. Would you like to provide more details about what you’re trying to achieve or any issues you’re encountering?
Thanks Matthew S.
You're welcome! If you have any more questions or need further assistance, feel free to ask. I'm here to help.
Its just the simplest way without really making a complicated AI based workflow will not work that well against a list that size
You're absolutely right - for a 6,700 row list, the sequential column approach is much more practical than complex AI workflows.
For lists that size, you might also want to consider our Bulk Enrichment feature, which is designed specifically for processing millions of records without the 50,000 row limit. It handles large datasets more efficiently than regular tables and automatically manages the processing flow. [1]
The sequential batching method you described keeps things simple and reliable while avoiding rate limits - definitely the right approach for your use case.
Did that answer your question?
I'm reading the instructions carefully, since I'm newbie in clay
That's great! Take your time getting familiar with the setup. The sequential batching approach we discussed is solid for beginners - it's straightforward and reliable.
Key things to remember as you set it up:
- Use "Only run if" conditions to create dependencies between columns [1]
- Test with just a few rows first before running your full 6,700 [2]
- The Clay.getCellStatus() formula gives you precise control over when each batch starts
If you get stuck on any part of the setup, just let me know what specific step you're working on and I can walk you through it.
