Building a Career Page Scraper in Clay with Claygent: Steps and Tips

I want to build a career page scrapper scenario in Clay. It should be with Claygent (AI Web researcher) to scrape career page? Can you share a loom on how to do that. Also, is there a way we can get this done in Clay without using credits.

6 comments

Arturo O.
·9 months ago·
Hey Noor! Thanks for reaching out about this use case. Happy to help. Of course, would like to better understand what data you're looking to scrape from those pages as it really depends on that when you're tasking Claygent to achieve this. We have this video about what Claygent could do and how to potentially structure your prompts to retrieve the information you're looking for. You can do this with or without Clay credits. Claygent allows you to bring your own API keys with most of the AI models offered in it, which allow you to use your own credits with OpenAI or Anthropic, for example. Alternative scraping tools that don't use AI but can still scrape given links are Zenrows, which you can also use your API key with, or Clay's Scrape Website integration too. Hope this helps for now!
Noor S.
·9 months ago·
Hey Arturo, I am looking to scrape jobs
Owen C.
·9 months ago·
Hey Noor, got it! I’d love to fully understand what success looks like here. A few questions come to mind: 1. Data to Scrape: Are you looking to scrape job titles, job descriptions, salary, other data points, or a combination of these? 2. Job Source: How do you plan on finding companies to target? 3. Why Clay? Have you considered using other scrapers to perform this action like Apify or Phantom Buster? (we integrate directly with these softwares) If so, what other data points, enrichment, or research are you hoping to get out of Clay? I’d love to help you brainstorm this! Just want to make sure I fully understand your situation, preferred setup, and use case.
Noor S.
·9 months ago·
1. Data to Scrape: Are you looking to scrape job titles, job descriptions, salary, other data points, or a combination of these? Yes, a combination of all. 2. Job Source: How do you plan on finding companies to target? Scrape websites like indeed, Glassdoor, LinkedIn to get the relevant job posting. 3. Why Clay? Have you considered using other scrapers to perform this action like Apify or Phantom Buster? (we integrate directly with these softwares) If so, what other data points, enrichment, or research are you hoping to get out of Clay? I've used Apify in the past but my leader wants me to use Clay for scrapping and later also document it how to use website scrapper in Clay. Maybe with claygent using Chatgpt option. Or something else that you way advice
Bo (.
·9 months ago·
Then in that case, you can use our “Find Jobs” feature. Here’s how it works: 1. Find Jobs While Creating a New Table • Use the Find Jobs Source Enrichment feature. • This option uses our sources directly and keeps things simple. 2. Find Jobs Waterfall from the Enrichment Panel • Click the “Add Enrichment” button at the top-right of the page. • Select the “Find Jobs” waterfall option. 3. Common Ways to Use It: • Option 1: • Create a new workbook and select “Find Jobs.” • Option 2: • Create a new workbook and select “Find Companies.” • Open a new workbook tab, then choose “Find Jobs” and apply it to the companies you found earlier. • Option 3: • Find companies first. • Go to the enrichment panel, click “Add Enrichment,” type “Find Jobs,” and select the “Jobs Waterfall” option. 4. Credits and Sources: • The “Jobs Waterfall” option checks multiple sources and uses credits for successful findings. • The “Find Jobs” option while creating a table checks only LinkedIn and is free. Let me know if you’d like more details or help setting this up! 😊
Channeled
APP
·9 months ago·
Hi Noor S.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:

Building a Career Page Scraper in Clay with Claygent: Steps and Tips | Clay