Dennis T. datalake sounds interesting; not too deep into this whole thing but consider interviewing some claygencies about that. as a small input: i would prefer to dump data once and get a solidified table back. because i need to do that once for a client where we take over because they did it themselves etc. thats what abdul said in the beginning. so its quite punctual. no sure if its a too big problem though. gtm engineers get paid to do it and often times you need to know whats inside this api blockbox. how it works etc. but datalake idea doesnt sound stupid, just need to be money validated.
I thought the same, remains the question if people are willing to pay for that - or building yourself is so cheap, that there is no reason to offer a service 😉
That’s actually a solid idea 👍 most teams I’ve seen end up hacking their own mini “data lake” with BQ or Airtable + Clay API to avoid re-enriching the same people/companies. having a central place to dump + dedupe before enrichment would save a lot of credits + headaches
Abdul S. I thought about creating a "data lake" for clay users. Where you simply dump everything into and later run an API call to check if you got this data point already (company , people). It's easy to do, but not sure if the problem is big enough
Exactly Dennis - re-enriching the same people/companies is brutal on credits that’s why I usually set up a ‘master deduped table’ in clay, then only enrich new rows against it saves credits + keeps things clean. Do you already have something like that set up or still running everything raw?
Abdul S. Wasting credits on results I already have. Companies for example - scraping the same company X times is "stupid" - same for individual people. 🙂
Do I get you right: If one field gets wrong (let's say an AI enrichment), the whole thing collapses?
Honestly, the most brutal task is reconciling messy, inconsistent data across multiple tables before any automation will actually run one wrong join or missing field and everything breaks it’s a total time sink even for experienced GTM engineers. what’s the task that drives you crazy? Dennis T.
.png){kind=logo}
