Inconsistent Claygent Results: Understanding 40% Variation Issues

Brett · 2025-06-11T14:04:18.868Z

We are having an issue with the consistency of claygent results. We run a claygent a cross a population of rows that visits the provided domain & performs a check for some specified characteristic (binary YES/NO) e.g. whether the business is a fund manager. When we re-run an identical duplicate of the same Claygent column, we consistently get different results - on average 40% variation. This seems deeply flawed? I'm certain other people must be encountering this. Any ideas why, and how does one address it? I have heard someone suggest it's due to the default temperature setting in the model but unsure (we can't make it more deterministic in settings). We typically use GPT-4.1-mini.

Inconsistent Claygent Results: Understanding 40% Variation Issues | Clay

8 comments

· Sorted by Oldest

Channeled
APP
·
·
The inconsistency you're seeing with Claygent results is likely related to the AI model you're using. GPT-4.1-mini, while cost-effective, may produce more variable results compared to other models. To improve consistency: 1. Consider switching to Claygent Neon, our flagship model specifically designed for agent tasks and data extraction. 2. Test your prompt on a small sample (about 10 rows) before running the full column to refine your instructions. 3. Make your search queries more specific and well-defined. 4. Iterate on your prompts and adjust wording to get more consistent results. Different models have different strengths - experiment to find what works best for your specific use case.
Sources: Claygent: AI Web Scraper [CRM Enrichment] - Clay University AI Formulas + Conditional Runs [Automated Inbound] - Clay University AI Formulas + Conditional Runs [Automated Outbound] - Clay University Guide: Ways to save Clay credits | Documentation | Clay University
Brett
·
·
Anyone else got ideas?
Channeled
APP
·
·
We have a few more ideas to improve consistency in Claygent results: 1. Use our limitless research feature with multiple enrichment tools for better data quality. 2. Try our AI-powered unstructured research capabilities for more accurate results. 3. Consider adjusting your input data or refining your search criteria for more consistent outcomes. What specific characteristic are you checking for in your Claygent column?
Source: What common use cases can I accomplish in Clay? - FAQ | Clay.com
Miguel M.
·
·
For me when its a task that requires more logic/thinking I tend to use Claude 3.5 Sonnet (more expensive than GPT 4.1 mini) but from the tests I have run its more accurate than any of the Claygent GPT versions (also Gemini). Claude 3.7 Sonnet provides the best results but its the most expensive (15 clay credits vs 3 clay credits for the 3.5 version). If you have an API from Anthropic and the right usage Tier should be quite cheap in comparison of using Clay credits.
Channeled
APP
·
·
For tasks requiring logic and accuracy, consider these approaches: 1. Use Claygent Argon for complex analysis (strongest overall model) or Claygent Neon for a good balance of performance and cost. 2. If using third-party models, Claude 3.5 Sonnet offers better accuracy than GPT models at a moderate cost (3 Clay credits vs 15 for Claude 3.7). 3. Improve consistency by: • Creating more specific, well-defined search queries • Testing prompts on small samples before full runs • Verifying input data format Different models have different strengths - experiment to find what works best for your specific use case.
Sources: Claygent: AI Web Scraper [Limitless Research] - Clay University Claygent: AI Web Scraper [Clay 101] - Clay University Claygent: AI Web Scraper [Automated Outbound] - Clay University
Miguel M.
·
·
I would recommend do small tests with a few of the Anthropic AIs and compare to GPT, Gemini and the native Clay ones to see which gives the best results
Channeled
APP
·
·
For more consistent Claygent results: 1. Choose the right model for your task: • Claygent Argon for complex analysis (strongest overall model) • Claygent Neon for data extraction and formatting • Claude 3.5 Sonnet for logic-heavy tasks at moderate cost 2. Improve your prompts: • Start with clear context and goals • Provide step-by-step instructions • Specify exact output format 3. Test on small samples before full runs to refine your approach 4. Use specific, well-defined search queries Experiment with different models to find what works best for your specific use case.
Sources: Claygent: AI Web Scraper [CRM Enrichment] - Clay University Claygent: AI Web Scraper [Clay 101] - Clay University Prompt Engineering Crash Course [Limitless Research] - Clay University
Brett
·
·
Thanks Miguel M.

Channeled
APP
·
·
The inconsistency you're seeing with Claygent results is likely related to the AI model you're using. GPT-4.1-mini, while cost-effective, may produce more variable results compared to other models. To improve consistency: 1. Consider switching to Claygent Neon, our flagship model specifically designed for agent tasks and data extraction. 2. Test your prompt on a small sample (about 10 rows) before running the full column to refine your instructions. 3. Make your search queries more specific and well-defined. 4. Iterate on your prompts and adjust wording to get more consistent results. Different models have different strengths - experiment to find what works best for your specific use case.
Sources: Claygent: AI Web Scraper [CRM Enrichment] - Clay University AI Formulas + Conditional Runs [Automated Inbound] - Clay University AI Formulas + Conditional Runs [Automated Outbound] - Clay University Guide: Ways to save Clay credits | Documentation | Clay University
Brett
·
·
Anyone else got ideas?
Channeled
APP
·
·
We have a few more ideas to improve consistency in Claygent results: 1. Use our limitless research feature with multiple enrichment tools for better data quality. 2. Try our AI-powered unstructured research capabilities for more accurate results. 3. Consider adjusting your input data or refining your search criteria for more consistent outcomes. What specific characteristic are you checking for in your Claygent column?
Source: What common use cases can I accomplish in Clay? - FAQ | Clay.com
Miguel M.
·
·
For me when its a task that requires more logic/thinking I tend to use Claude 3.5 Sonnet (more expensive than GPT 4.1 mini) but from the tests I have run its more accurate than any of the Claygent GPT versions (also Gemini). Claude 3.7 Sonnet provides the best results but its the most expensive (15 clay credits vs 3 clay credits for the 3.5 version). If you have an API from Anthropic and the right usage Tier should be quite cheap in comparison of using Clay credits.
Channeled
APP
·
·
For tasks requiring logic and accuracy, consider these approaches: 1. Use Claygent Argon for complex analysis (strongest overall model) or Claygent Neon for a good balance of performance and cost. 2. If using third-party models, Claude 3.5 Sonnet offers better accuracy than GPT models at a moderate cost (3 Clay credits vs 15 for Claude 3.7). 3. Improve consistency by: • Creating more specific, well-defined search queries • Testing prompts on small samples before full runs • Verifying input data format Different models have different strengths - experiment to find what works best for your specific use case.
Sources: Claygent: AI Web Scraper [Limitless Research] - Clay University Claygent: AI Web Scraper [Clay 101] - Clay University Claygent: AI Web Scraper [Automated Outbound] - Clay University
Miguel M.
·
·
I would recommend do small tests with a few of the Anthropic AIs and compare to GPT, Gemini and the native Clay ones to see which gives the best results
Channeled
APP
·
·
For more consistent Claygent results: 1. Choose the right model for your task: • Claygent Argon for complex analysis (strongest overall model) • Claygent Neon for data extraction and formatting • Claude 3.5 Sonnet for logic-heavy tasks at moderate cost 2. Improve your prompts: • Start with clear context and goals • Provide step-by-step instructions • Specify exact output format 3. Test on small samples before full runs to refine your approach 4. Use specific, well-defined search queries Experiment with different models to find what works best for your specific use case.
Sources: Claygent: AI Web Scraper [CRM Enrichment] - Clay University Claygent: AI Web Scraper [Clay 101] - Clay University Prompt Engineering Crash Course [Limitless Research] - Clay University
Brett
·
·
Thanks Miguel M.