Clay Icon

Improving GPT Accuracy for Job Vacancy Classification in Clay

·
·

How to avoid 50-score inconsistencies when classifying job vacancies with GPT in Clay? I'm using Clay with GPT-4o to analyze job vacancy pages and determine how likely the role is to be for a customer service position at an online webshop vs. a brick-and-mortar store. Here’s the exact prompt I’m using: “Analyze /url Assignment: - You are an AI that analyzes job vacancy texts and determines how likely the job is for a customer service position at an online webshop versus a brick-and-mortar store. Return only a single number from 1 to 100, where: 1 = Definitely a job at a brick-and-mortar store 100 = Definitely a job at an online webshop Use clues in the text such as: - Brick-and-mortar indicators (score 1–33): "in store", "at location", "in the office", "local", "retail store", "shop", "physical store", "cash register", "face-to-face", "onsite", "field work", ''brick and mortar'', ''in-store'', ''brick & mortar'', ''instore'', ''on site'' - Hybrid indicators (score 34–66): "partially digital", "hybrid", "semi digital", "part digital", "limited digital", "multi-location", "partially remote", "store and e-commerce", "cross-channel" - Online webshop indicators (score 67–100): "remote", "online", "work from home", "e-commerce", "webshop", ''ecommerce'', ''ecom'', "digital", "chat/email", "virtual", ''home-office'', ''home office'' Scoring mechanism: - If multiple types of indicators are present, assign a middle-range score accordingly. If the text is vague, score based on the most probable context. Exception: - if you're not certain and give it a score of 50, do a second analysis by analyzing /description Output: - Return only a number. Do not include any explanation or text.The issue: When I run this formula across the full table, many rows return a score of exactly 50, marked with a red square in Clay—meaning the model wasn’t confident. But when I run those exact same rows manually or one-by-one, GPT almost always gives a more accurate score the second time (e.g., 22, 87, etc.). My question: Is there a way to improve this so GPT gets it right the first time during the bulk run? I’d love to avoid manually re-running 50s row-by-row after the fact. Has anyone dealt with this and found a reliable workaround? Open to suggestions around retries, batching, better prompting structure, or any Clay-specific setting that might help.

  • Avatar of Channeled
    Channeled
    APP
    ·
    ·

    To improve GPT's accuracy in classifying job vacancies during bulk runs in Clay, consider these strategies: 1. Refine your prompt: Include more specific indicators and adjust the scoring mechanism to reduce ambiguity. 2. Use company enrichment: Gather additional industry information to provide more context for classification. 3. Leverage the "Job Openings" waterfall: This combines data from multiple sources, potentially offering more accurate information for classification. 4. Add relevant keywords: Include specific job-related terms to help narrow down the classification. 5. Consider using our "write to table" feature to store and analyze results more effectively. If issues persist, you may need to adjust your AI model or data processing approach. For complex cases, consider breaking down the classification into multiple steps or using a combination of AI and rule-based methods.

  • Avatar of Muhammad S.
    Muhammad S.
    ·
    ·

    Yes, you just need to make the prompts and instructions more simpler and add some training examples. The easier the prompt the better the results

  • Avatar of Channeled
    Channeled
    APP
    ·
    ·