Web Scraper Chrome Extension: How to Automate Data Extraction Steps

Hello friends! My question is regarding the webscraper via the Chrome extension. Is it possible for the scraper to do this:

1.
Select the 1st county from the list and click go
2.
Click on the school profile link
3.
Scrape data

I've included images of what I want to scrape in the order in how I want to scrape them.

10 comments

Channeled
APP
·11 months ago·
👋 Hey there! Our support team has got your message - we'll be back in touch within 24 hours (often sooner!). If you haven't already, please include the URL of your table in this thread so that we can help you as quickly as possible!
Addison F.
·11 months ago·
Going to use this data to populate a table, so I don't have a link yet 🙂
Bo (.
·11 months ago·
Hey, thanks for reaching out! No, our Chrome extension can scrape page by page but requires manual changes between pages. Once you’ve collected the full directory, you can use our “map websites” feature to map the subpages and pull that data into Clay using the “find data from page” function. Let me know if you need help setting that up! 😊
Addison F.
·11 months ago·
Thanks Bo (.! So you're saying if I map this page out, I can use the links to pull in data from those pages, so long as I map out the data I want from the individual pages?
Daniela D.
·11 months ago·
Hey Addison! Thanks for reaching out. Jumping in for Bo. Here's a quick breakdown: 1. To extract the data in the first screenshot, you'll need to first scrape the directory or list page by page. This sometimes is auto-mapped in the extension and can be imported into a Clay table after extracting all pages or you can follow this guide to map those fields: https://www.loom.com/share/e5916863f02c480aa4d2d8c3e0b4b287 2. Once the records are imported into a Clay table, select one of the sub-pages. On that example sub-page, use the extension to create a recipe & map the fields you need to extract using this guide: https://www.loom.com/share/31a83c83ce794dfa86a6a91a27d7dd08 https://docs.clay.com/en/articles/9672330-find-data-from-page-with-clay-scrapers 3. Once the recipe is created, you can add the "find data from page" column to your table and map the subpage links to scrape. Once that is done, run the integration. The recipe created from mapping that one sub-page will apply to all sub pages in the table. Let us know if you run into any issues. https://downloads.intercomcdn.com/i/o/1178293522/609ee166dc8241759a3d65b9/CleanShot+2024-09-12+at+21_25_16%402x.png?expires=1726174800&signature=fa237bea930a4a88d4447e59d9b6770783764c497bb89f4c5b260203a412ebb1&req=dSEgHst3noRdW%2FMW1HO4zWzr55yRGkkKqJpfuPlW1Qv%2FDjqrzbsNef9JPlyu%0AjeyF%0A https://downloads.intercomcdn.com/i/o/1178289302/975a0a90b433c3c67335aced/CleanShot+2024-09-12+at+21_21_30%402x.png?expires=1726174800&signature=31adb7ac201deb084ae180c8fe54562ce1302dfd4a3b9d13a1f058466de8ae07&req=dSEgHst2lIJfW%2FMW1HO4zUOUYGb%2BAliEkx6kC5ujsj58gh4pxTHsTeAe1k9V%0AJC6g%0A
Addison F.
·11 months ago·
Hi Daniela D. thanks for this! I went ahead and scraped all of the pages with a recipe but it added all my scrapes into different tables. Any way to merge or have the scrape go into one table?
Bo (.
·11 months ago·
Hey, thanks for reaching out! 😊 To merge your tables or ensure future scrapes go into one table, you can use the “Write to Table” feature to combine data from multiple tables. This will work for all future scrapes as well. Here’s how to do it with lists: Video with Lists And here’s how to do it without lists: Video without Lists Let me know if you need any help! 🙂
Channeled
APP
·11 months ago·
We haven't heard back from you in a bit, so we're going to go ahead and close things out here - feel free to let us know if you still need something!
Channeled
APP
·11 months ago·
Hi Addison F.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:
Channeled
APP
·10 months ago·
Thank you so much for sharing your feedback Addison F.!

Web Scraper Chrome Extension: How to Automate Data Extraction Steps | Clay