Strategies for Scraping Job Data from Various Sources

Elric L. · 2024-11-12 22:15:02.913314+00

I’m working on scoring 1,300 accounts (in different industries). I’m having trouble with one specific score: job data. Specifically, I’m trying to scrape job data from various sources (LinkedIn, Indeed, and company career pages) to gather information like the if they have a career page, ATS used, current number of job openings, posting dates, and an estimate of the percentage of in-office versus operational roles. How would you approach this? One option might be to start with the job listings with Clay + Apify on their career page, then check LinkedIn, and finally check Indeed. Curious about your thoughts!

Strategies for Scraping Job Data from Various Sources | Clay

10 comments

· Sorted by Oldest

Tanvi R.
·
·
Hi Elric, thanks for reaching out and welcome to Clay! Happy to help you approach this workflow ✨
Tanvi R.
·
·
Here's a tutorial I made for you on how to approach this: https://www.loom.com/share/b0f2fb9c34724589818c7bf53b415e0b?sid=717c7d03-8c72-4f5d-83e9-b882990c25ca
Tanvi R.
·
·
Generally, I recommend starting with the career page job listings and then enriching the company information to build your scoring criteria. I broke down each of these steps for you in my video. Let me know if you have any other questions! 😊
1
1
Elric L.
·
·
Tanvi R. thank you so much! That was quick and super helpful. One follow-up question regarding in-office vs. other roles. I’m not looking to differentiate between in-office and remote work. Instead, I need to determine the percentage of roles that are based in an office (like sales, HR, accounting) versus roles in locations like factories, warehouses, retail stores, etc. This distinction is important because the ICP is companies with primarily in-office roles, rather than those hiring for blue-collar positions like drivers or warehouse staff.
Tanvi R.
·
·
Got it Elric, thanks for clarifying! I didn't realize that was what you're looking for, but you can just input this into your Claygent AI prompt and use the metaprompter to write the most effective prompt for you here! The same approach of using Claygent will apply here regardless
Tanvi R.
·
·
Let me know if this makes sense and if you have any other questions!
1
Elric L.
·
·
thank you, Ill give it a try today
Tanvi R.
·
·
Sounds great, keep me posted! Additionally, you can put all of this different criteria together into a Lead score by using theScore Row in Clayenrichment. Happy to help you get this set up if you have any questions here!
Channeled
APP
·
·
Hi Elric L.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:
Channeled
APP
·
·
Thank you so much for sharing your feedback Elric L.!

Tanvi R.
·
·
Hi Elric, thanks for reaching out and welcome to Clay! Happy to help you approach this workflow ✨
Tanvi R.
·
·
Here's a tutorial I made for you on how to approach this: https://www.loom.com/share/b0f2fb9c34724589818c7bf53b415e0b?sid=717c7d03-8c72-4f5d-83e9-b882990c25ca
Tanvi R.
·
·
Generally, I recommend starting with the career page job listings and then enriching the company information to build your scoring criteria. I broke down each of these steps for you in my video. Let me know if you have any other questions! 😊
1
1
Elric L.
·
·
Tanvi R. thank you so much! That was quick and super helpful. One follow-up question regarding in-office vs. other roles. I’m not looking to differentiate between in-office and remote work. Instead, I need to determine the percentage of roles that are based in an office (like sales, HR, accounting) versus roles in locations like factories, warehouses, retail stores, etc. This distinction is important because the ICP is companies with primarily in-office roles, rather than those hiring for blue-collar positions like drivers or warehouse staff.
Tanvi R.
·
·
Got it Elric, thanks for clarifying! I didn't realize that was what you're looking for, but you can just input this into your Claygent AI prompt and use the metaprompter to write the most effective prompt for you here! The same approach of using Claygent will apply here regardless
Tanvi R.
·
·
Let me know if this makes sense and if you have any other questions!
1
Elric L.
·
·
thank you, Ill give it a try today
Tanvi R.
·
·
Sounds great, keep me posted! Additionally, you can put all of this different criteria together into a Lead score by using theScore Row in Clayenrichment. Happy to help you get this set up if you have any questions here!
Channeled
APP
·
·
Hi Elric L.! This thread was recently closed by our Support team. If you have a moment, please share your feedback:
Channeled
APP
·
·
Thank you so much for sharing your feedback Elric L.!