Hey guys, Google Sheets resulted in error in thousands of rows in this table - can you check what's the root cause? https://app.clay.com/workspaces/105783/workbooks/wb_uDC7scm3BYsj/tables/t_PdPF9mgTN8fm/views/gv_CxpxXE8AeMUt
Hi Leon! Happy to help! It seems the errors are related to Google's rate limits. Google API is known for its low rate limit, around 60 requests per minute per project, so this is completely normal. Unfortunately, there’s not much we can do to increase that limit. A great way to work around this is to choose a certain number of rows at a time. Alternatively, have you considered using no-code databases with more flexible API limits? Here are a few options that I would recommend: - NocoDB - Seatable.io - Baserow.io - Gigasheet - Airtable - Xano (more technical) - Supabase (more technical) Additionally, some of these solutions can also be self-hosted. Let me know and I'm happy to recommend a self-hosting solution. Hope this helps! :) Let me know if you need more help! https://downloads.intercomcdn.com/i/o/w28k1kwz/1241725707/6a9411c958f56532f58084e173a1/CleanShot-2B2024-09-16-2Bat-2B_215k8S6miH-402x.png?expires=1730898900&signature=15dbd7bd326da18ebadef9975d8251bfda8dcb0d1a1d6c886596dd0d5290f13e&req=dSIjF858mIZfXvMW1HO4zZvp72allQFXN5E09GYzYYXtS6Cfo98QUuFnZ6D%2F%0AAa8T%0A https://downloads.intercomcdn.com/i/o/w28k1kwz/1241725708/26a9b948c0c7ac0326761457cf43/CleanShot%2B2024-10-04%2Bat%2B17_28_06-402x.png?expires=1730898900&signature=342b6602c3602ea0ced7aeb9855fd6d641a385c86a489324c8568c58c79fa0d5&req=dSIjF858mIZfUfMW1HO4zWUdwYrrI3nH0BYgmJE2QqLnZkx14AOX1s%2BV8pAY%0Ayi%2B%2B%0A
Thanks! We've reopened this thread. You can continue to add more detail directly in this thread.
Hey Daniel, would you be able to tell me why google sheets api returns error after waiting in queue for a long time vs e.g. HubSpot that can manage multiple rows without erroring. Do you have any documentation around what constitues an error vs retrying vs not run? I'm trying to work with the inherent limitations and it would be useful to understand how they work on the backend in clay
Hi Daniel, Thanks for your question! Could you share the table URL with us so we can take a closer look? Here’s some context about why this might be happening: Google Sheets API has stricter rate limits compared to some other integrations like HubSpot. When processing large volumes of data, the API may time out or hit these limits, causing errors after long queue times. HubSpot, on the other hand, is optimized to handle bulk operations more efficiently, which is why it may manage multiple rows without issue. If you’d like, I can also share details about what constitutes an error, a retry, or a non-run. Let me know, and I’d be happy to help further! 😊
is Bo a bot?
Hehe, I wish I was! 😎 Happy Saturday! I’m not able to see the error on the backend right now, but I’ll need to check with our team to get more details on this. I’ll keep you updated!
no need to run these rows, just need to learn what triggers errors with google sheets integration if possible
In the meantime, I’d suggest letting it run for a while since Google Sheets has a very slow rate limit. This might resolve the issue over time. Apologies for the inconvenience!
Got it! I’ll wait to hear back from the team before responding, as I’m not entirely sure why these errors are happening at the moment. 😊
yeah see I don't get why some integrations in clay can auto rerun and all is gucci while others seem to crash. Maybe that's ludicrous to ask for internal docs on that but do you have any sense on why auto-rerunning columns only work with certain integrations?
The auto-update might be off on those that aren't running. There's two auto-update. One for the table, and one for each integration, you can still turn off integrations alone. That might be why. Which columns would you say you're having issues with? https://downloads.intercomcdn.com/i/o/w28k1kwz/1245982010/de5de2be7ff32b5d7d94ef8c6f5f/CleanShot+2024-11-09+at+_39r1hiMkRC%402x.png?expires=1731183300&signature=bab8899104032f9d8f4a19ce5b3694ce1deabb958e943dd06bdb1ca1b53137b1&req=dSIjE8B2n4FeWfMW1HO4zd1pkG4bCctbopket001n0UQ4CJiRZHWM3tL%2BH7e%0AipyW%0A
yeah the conditions are met for all and auto-update is on, howver once a column errors it won't rerun after some time with google sheets
You're right, auto-update won't re-run errored row automatically. They need to be re-updated or have their dependencies updated with the auto-update on so it re-runs!
is there a way to fix this for google sheets?
I'm still waiting to hear back what is causing it. Once we know, we'll be able to work on a fix here, Leon! :)
ah, fair 😂 thanks a lot!
actually, I've got one specific use case for google sheets that maybe doable otherwise without diving too deep, I want to reference the output of a column in an api call. this column is going to be plain text where I use dynamic variables in its formula and then reference the plain text (after processing) in the api call. Would I be able to instead input dynamic variables into the value of a key value pair in a http request? E.g. "prompt": JSON.stringify("Hey {{first_name}}, this is a HTTP request") Is there a way to make it work vs referencing the pure column in a HTTP request?
Hi there, Are you trying to pass "prompt": "JSON.stringify(\"Hey {{first_name}}, this is a HTTP request\")" as a value? To validate JSON, I’d recommend using a tool like JSONLint. It helps ensure the JSON is structured correctly. If you’re including JSON.stringify directly in the value, it won’t work as JSON—it’s more suited for JavaScript. You’d need to handle it differently depending on the use case. Could you clarify what you’re trying to achieve here? That will help me guide you better! 😊
Are you trying to pass "prompt": JSON.stringify("Hey {{first_name}}, this is a HTTP request")" as a value? If so that won't work as you can see in the screenshot To validate JSON, I’d recommend using a tool like JSONLint. It helps ensure the JSON is structured correctly. If you’re including JSON.stringify directly in the value, it won’t work as JSON—it’s more suited for JavaScript. You’d need to handle it differently depending on the use case. Could you clarify what you’re trying to achieve here? That will help me guide you better! 😊
I'm basically looking to replace Clay.formatForJSON for Clay referencing a value of a column with a way to do it myself within the API call
Hey Leon, the easiest way would be create an AI formula, create all the data (whether dynamic or not) and inputs there and map it to the HTTP body text. Have you tried that yet?
yes, the only issue is that if this exceeds 8k characters, it won't work
Leon, to overcome the 8,000 character limit per cell in Clay, you can split the text into two or more columns. This method allows you to divide the text across multiple columns and manage larger amounts of data within the cell size constraints of Clay. Can you let me know if this would work? If it doesn't, feel free to share a link to your table and pinpoint me to the section within your table and I'll have a look. We're going to make it work for you :)
.png){kind=logo}
