Clay Icon

Implementing Deduplication on Multiple Columns in Apify Jobs Data

ยท
ยท

Hey guys, is there a way to put deduplication on two columns? I'm bringing in open jobs from multiple job boards using Apify but there has been a lot of duplication recently. Some companies have multiple jobs open as well. Currently, the duplication check is on job url column but ideally, Iยดd like to check company name as well. Any guidance here would be appreciated. https://app.clay.com/workspaces/286951/tables/t_xZk5sDMNmmPk/views/gv_EoYse8z7Py8u

  • Avatar of Channeled
    Channeled
    APP
    ยท
    ยท

    Currently, Clay only allows auto-deduplication on a single column at a time. Here's how you can enable it:1. Click the Auto-dedupe icon at the bottom right of your table 2. Select "Enable automatic deduplication" 3. Choose your preferred column (job URL in your case) 4. Click "Save Changes"While multi-column deduplication isn't available yet, auto-dedupe helps keep your table clean by deleting duplicates and saves credits by preventing the same entries from being enriched multiple times.For your specific case, I recommend using the job URL for deduplication since it's likely to be the most unique identifier. If you need more sophisticated deduplication, you may want to request human support to explore alternative approaches. Sources: Auto-update and Auto-dedupe Table Auto-dedupe

  • Avatar of Channeled
    Channeled
    APP
    ยท
    ยท

    Hi Reyhan K.! This thread was recently closed by our Support team. If you have a moment, please share your feedback: