Clay Icon
Join the Community
02 Support

Troubleshooting AI Mismatch: Understanding Incorrect Results in Clay's AI Option

Avatar of ArnaudArnaud
·
·

Hi there, I used the AI option in Clay with a prompt that runs on all my DB (± 4200 lines with GPT3.5 and my API key). The prompt is simple: from a name, tell me if there is a match with one of the 3 offered options. It seemed to work well and gave me all the results, but I realized that the answer was incorrect for many of them. And when I try again by clicking manually on each one, it gives me the correct answer. How can we explain that? Thanks a lot!

13 comments

  • Avatar of Arnaud
    Arnaud
    ·
    ·

    For a concrete example you can try to run again line 931 (163401 with Ctrl+F)

  • Avatar of Arnaud
    Arnaud
    ·
    ·

    Column "Company Match" (you can use my own credits)

  • Avatar of Arnaud
    Arnaud
    ·
    ·

    Some other examples (you can use Ctrl+F to find them):

    • 38501

    • 138203

  • Avatar of Arnaud
    Arnaud
    ·
    ·

    Here the answer should be 1 And if you run it again, you will have the correct answer

  • Avatar of Clay Team
    Clay Team
    APP
    ·
    ·

    Hey Arnaud!!

  • Avatar of Clay Team
    Clay Team
    APP
    ·
    ·

    Took a deeper look at this but thinking that since it's AI, it may have misunderstood the prompt in some of these rows. Perhaps if it's given a few examples it may be a lot better at determining what a match looks like though.

  • Avatar of Arnaud
    Arnaud
    ·
    ·

    Hey Arturo O.! It's weird because if it didn't understand it, it would always give the wrong answer, no? But here, it seems that it only makes mistakes when it's done in bulk (it's always correct when I do it individually)

  • Avatar of Arnaud
    Arnaud
    ·
    ·

    Don't you think that doing it in bulk has an impact on the AI performance?

  • Avatar of Clay Team
    Clay Team
    APP
    ·
    ·

    Sorry for the delay, Arnaud!

  • Avatar of Clay Team
    Clay Team
    APP
    ·
    ·

    Perhaps, I've seen similar reports before where people run calls in bulk so not sure if the AI's server just can't process some of them accordingly and runs into this issue. I do find that providing examples tends to help decrease these instances though, but there are always edge cases.As AI gets a lot better, this may not be an issue in coming months. 🤞

  • Avatar of Arnaud
    Arnaud
    ·
    ·

    Ok thanks for the feedback Arturo O.! And if I add some examples, will it be more expensive to run if I use my own API? (does it make the prompt longer?) Thanks!

  • Avatar of Clay Team
    Clay Team
    APP
    ·
    ·

    It does make the prompt slightly longer but since it's just a few lines of text, the token count increase should be very minimal, which should only be a few cents.-> OpenAI's token calculator: https://platform.openai.com/tokenizer