Home
Events
Groups
Members

00 Clay Announcements
01 Community General
01 Props and Wins
01 Watercooler
01 Welcome and Intros
02 Bugs
02 Starting with Clay
02 Support
03 Feedback & Requests
03 Tutorials & Inspiration
04 FT Job Opps

Community Guidelines

Powered by Tightknit

Web Scraper Chrome Extension: How to Automate Data Extraction Steps

·

·

Hello friends! My question is regarding the webscraper via the Chrome extension. Is it possible for the scraper to do this:

1.
Select the 1st county from the list and click go
2.
Click on the school profile link
3.
Scrape data

I've included images of what I want to scrape in the order in how I want to scrape them.

Loading attachments

Web Scraper Chrome Extension: How to Automate Data Extraction Steps | Clay

10 comments

· Sorted by Oldest

Channeled
APP
·
·
👋 Hey there! Our support team has got your message - we'll be back in touch within 24 hours (often sooner!). If you haven't already, please include the URL of your table in this thread so that we can help you as quickly as possible!
Addison F.
·
·
Going to use this data to populate a table, so I don't have a link yet 🙂
Bo (.
·
·
Hey, thanks for reaching out! No, our Chrome extension can scrape page by page but requires manual changes between pages. Once you’ve collected the full directory, you can use our “map websites” feature to map the subpages and pull that data into Clay using the “find data from page” function. Let me know if you need help setting that up! 😊
Addison F.
·
·
Thanks Bo (.! So you're saying if I map this page out, I can use the links to pull in data from those pages, so long as I map out the data I want from the individual pages?
Daniela D.
·
·
Hey Addison! Thanks for reaching out. Jumping in for Bo. Here's a quick breakdown: 1. To extract the data in the first screenshot, you'll need to first scrape the directory or list page by page. This sometimes is auto-mapped in the extension and can be imported into a Clay table after extracting all pages or you can follow this guide to map those fields: https://www.loom.com/share/e5916863f02c480aa4d2d8c3e0b4b287 2. Once the records are imported into a Clay table, select one of the sub-pages. On that example sub-page, use the extension to create a recipe & map the fields you need to extract using this guide: https://www.loom.com/share/31a83c83ce794dfa86a6a91a27d7dd08 https://docs.clay.com/en/articles/9672330-find-data-from-page-with-clay-scrapers 3. Once the recipe is created, you can add the "find data from page" column to your table and map the subpage links to scrape. Once that is done, run the integration. The recipe created from mapping that one sub-page will apply to all sub pages in the table. Let us know if you run into any issues. https://downloads.intercomcdn.com/i/o/1178293522/609ee166dc8241759a3d65b9/CleanShot+2024-09-12+at+21_25_16%402x.png?expires=1726174800&signature=fa237bea930a4a88d4447e59d9b6770783764c497bb89f4c5b260203a412ebb1&req=dSEgHst3noRdW%2FMW1HO4zWzr55yRGkkKqJpfuPlW1Qv%2FDjqrzbsNef9JPlyu%0AjeyF%0A https://downloads.intercomcdn.com/i/o/1178289302/975a0a90b433c3c67335aced/CleanShot+2024-09-12+at+21_21_30%402x.png?expires=1726174800&signature=31adb7ac201deb084ae180c8fe54562ce1302dfd4a3b9d13a1f058466de8ae07&req=dSEgHst2lIJfW%2FMW1HO4zUOUYGb%2BAliEkx6kC5ujsj58gh4pxTHsTeAe1k9V%0AJC6g%0A
Addison F.
·
·
Hi Daniela D. thanks for this! I went ahead and scraped all of the pages with a recipe but it added all my scrapes into different tables. Any way to merge or have the scrape go into one table?
Bo (.
·
·
Hey, thanks for reaching out! 😊 To merge your tables or ensure future scrapes go into one table, you can use the “Write to Table” feature to combine data from multiple tables. This will work for all future scrapes as well. Here’s how to do it with lists: Video with Lists And here’s how to do it without lists: Video without Lists Let me know if you need any help! 🙂
Channeled
APP
·
·
We haven't heard back from you in a bit, so we're going to go ahead and close things out here - feel free to let us know if you still need something!
Channeled
APP
·
·
Hi Addison F.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:
Channeled
APP
·
·
Thank you so much for sharing your feedback Addison F.!