Clay Icon

Troubleshooting Claygent to Scrape Multiple Real Estate Agents' Data

·
·

I am trying to scrape data from a website which has a list of real estate brokers, in my Claygent prompt, I have given the instruction to scrape all the agents that are listed but Claygent is only scraping the details of one. Any idea what I might be doing wrong Link to the clay table is appended below for reference- https://app.clay.com/workspaces/256831/tables/t_nBBogqtnpqxc/views/gv_vcUcXAnfinCF

  • Avatar of Channeled
    Channeled
    APP
    ·
    ·

    💤 You've caught us outside of our support hours (9am-9pm EST), but don't worry - we'll be back in touch within 24 hours (often sooner!). If you haven't already, please include the URL of your table in the thread below so that we can help you as quickly as possible!

  • Avatar of Bo (.
    Bo (.
    ·
    ·

    Hey, thanks for reaching out! 😊 To scrape an entire webpage with all the data, you’ll need to use a tool like Scrape Website, not Claygent. Claygent is great for extracting information in smaller batches or from specific inputs, but it’s not ideal for scraping an entire directory. Here’s how we typically see people handling this: 1. Use a Chrome Extension to scrape the website. 2. Map the website data you need. 3. Export the results into your table. 4. Use Find Data From Page to pull data from sub-pages for more detailed information. Here are some helpful links for reference: • Find Data From Page with Clay ScrapersScrape Website DocumentationChrome Extension Let me know if you need any further guidance! 😊

  • Avatar of Mrinal S.
    Mrinal S.
    ·
    ·

    Thanks Bo, but seems that Clay chrome extensions link is not working

  • Avatar of Bo (.
    Bo (.
    ·
    ·

    Curious what are you seeing? If you're having any issues, You can use the instructions here to get started with the beta - https://clayrun.notion.site/Clay-Chrome-Extension-Beta-52bab9914afd4a448887fab4980bfa7f. There's also a channel in the community Slack, #chrome-ext-beta, in case you need any help! Let me know if you have any feedback,

  • Avatar of Mrinal S.
    Mrinal S.
    ·
    ·

    Trying the new extension

  • Avatar of Bo (.
    Bo (.
    ·
    ·

    Ok perfect - Keep us posted :)

  • Avatar of Mrinal S.
    Mrinal S.
    ·
    ·

    Still no luck

  • Avatar of Mrinal S.
    Mrinal S.
    ·
    ·

    the issue might be with Justdial.com

  • Avatar of Bo (.
    Bo (.
    ·
    ·

    Could you clarify what you mean by “no luck”? It sounds like you’re trying to access a URL, but you’re encountering an issue. Can you try with another URL and see if that works? Also, what’s your goal here exactly? I’d be happy to help troubleshoot and guide you accordingly! 😊 https://downloads.intercomcdn.com/i/o/1180232989/7096a26466aab25f0e0f35e3/CleanShot+2024-09-14+at+_24RNw7vMfS%402x.png?expires=1726344900&signature=19abd0ecbffc962a18abf8da913972283cdb0109609c476598e8ef31c4ec14a2&req=dSEvFst9n4hXUPMW1HO4zSWUwMO6nLlBj6Qtbd3qrFCyRfjcxCjT6sL%2FtUhs%0AGYqJ%0A

  • Avatar of Channeled
    Channeled
    APP
    ·
    ·

    We haven't heard back from you in a bit, so we're going to go ahead and close things out here - feel free to let us know if you still need something!

  • Avatar of Channeled
    Channeled
    APP
    ·
    ·

    Hi Mrinal S.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:

  • Avatar of Mrinal S.
    Mrinal S.
    ·
    ·

    Hi Bo, I was trying to scrape the name and phone numbers from this site, idea was to see if a Clay table be designed to run a call and sms based campaign

  • Avatar of Channeled
    Channeled
    APP
    ·
    ·

    Thanks! We've reopened this thread. You can continue to add more detail directly in this thread.

  • Avatar of LuisArturo
    LuisArturo
    ·
    ·

    Hey there jumping in for Bo, after some testing on our end, it looks like the website is preventing Clay's scrappers from running in the webpages. You can try to see if Apify, Phantombuster, or Instant Data Scrapper can scrape the page for results. If using Apify or Phantombuster you can make use of integrations with either platform to bring results into a Clay table. And for Instant Data Scrapper you can download the results into a CSV and then upload that to Clay, since we do not have a integration with them.

  • Avatar of Channeled
    Channeled
    APP
    ·
    ·

    Hi Mrinal S.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:

  • Avatar of Channeled
    Channeled
    APP
    ·
    ·

    Thank you so much for sharing your feedback Mrinal S.!