Why AI Metadata Enrichment Accuracy Drops When Processing Large Batches
I've developed a working / accurate AI prompt for company metadata enrichment - when I run it one company or even 10 companies at the time the accuracy is great. When I run 400 companies at once - it grades everyone an A in the two dimensions of a rubric I prompted it to grade - highly inaccurate, and different from the data when pulling 1-10 companies through the enrichment at a time.
- 1.
Why is this?
- a.
Assuming Rate limiting or some type of batch processing defects.
- 2.
Are there any suggested best practices to mitigate this?
- a.
For now i will batch by 10, 20, 30 and see which of these thresholds achieves good tradeoff between batch size and accuracy .... but I have 65k CO.'s to enrich (will do it 10 at a time 6500 times if I have too.... but preferably not!)
