Hi everyone!
Does anyone here have experience with Clay’s custom Claygent builder?
I built a custom research agent designed to scan a company’s website and identify specific financial features. I defined each feature category very clearly, provided explicit examples for every variable, and during initial test runs the agent performed flawlessly. It correctly detected branded payments, accounting integrations, reporting, etc.
But as we’ve started using it more, the accuracy and confidence have noticeably deteriorated. For instance, instead of identifying the branded payments product (like “XPay” or “Company Payments”), it has started outputting the underlying payment processor instead, which directly contradicts the instructions we gave it. It’s now ignoring parts of the prompt that it initially followed perfectly.
We’re currently running it on the Argon model, so I’m also wondering whether switching to a different model might improve stability or reduce this kind of drift.
If anyone has seen similar behavior with custom agents or has advice on prompt design, model selection, or keeping agents aligned with their instructions over time, I’d love any tips or guidance. Thanks!
