Clay Icon
Join the Community
02 Support

Optimizing Web Scraping: How to Specify Part of a Homepage for Efficient Results

Avatar of Gilbert K.Gilbert K.
·
·

How can I specify which part of a homepage to scrape - seems like the inital page size is to big to even get a results from the scraper.

5 comments

  • Avatar of Clay Team
    Clay Team
    APP
    ·
    ·

    Hey Gilbert, thanks for reaching out! We don't currently have a way to limit how much it scrapes with this integration but it does have the option to specify what fields you'd like it to return. The issue is that the body of text still tends to be excessively large even when you select it as the only output, but it could help in some cases

    5c1bba7a-763a-4e95-91f0-7a68f20fed69.png(39 kB)
  • Avatar of Clay Team
    Clay Team
    APP
    ·
    ·

    The team is already looking into the cell size data issue, so a few options coming.

  • Avatar of Gilbert K.
    Gilbert K.
    ·
    ·

    Yes, i tried to body text but the page seems to be too big so I was looking to slice before the output happens

  • Avatar of Gilbert K.
    Gilbert K.
    ·
    ·

    Looking forward to the fix