⚡ Scraping Websites just got better ⚡

Eric E. · 2023-11-14 19:26:46.514206+00

:zap: *Scraping Websites just got better* :zap: Happy Friday, Everyone! I just shipped a change to Clay’s Scrape website integration that makes scraping more reliable and also enables some new features. We’ve added: • The ability to delay results (for websites that take a long time to load) • The ability to add custom regex to find things we might not track natively (like Wikipedia URLs) • The ability to automatically retry when a scrape fails to make sure you’re getting the results you need We’ve also fixed a few issues with `www.` being removed when scraping and not rendering Javascript properly when trying to find the body text. Please let me know if you have any issues with our `Scrape Website` action, and I’m excited for you all to see the improvements we have in the pipeline as well! This is only the beginning :wink:.

Eric E.

⚡ Scraping Websites just got better ⚡ Happy Friday, Everyone! I just shipped a change to Clay’s Scrape website integration that makes scraping more reliable and also enables some new features. We’ve added:

The ability to delay results (for websites that take a long time to load)
The ability to add custom regex to find things we might not track natively (like Wikipedia URLs)
The ability to automatically retry when a scrape fails to make sure you’re getting the results you need

We’ve also fixed a few issues with www. being removed when scraping and not rendering Javascript properly when trying to find the body text. Please let me know if you have any issues with our Scrape Website action, and I’m excited for you all to see the improvements we have in the pipeline as well! This is only the beginning 😉.

6 comments