.png){kind=logo}
Scraping Blog Titles from Websites Using Clay
I have a data of website url and I want to see if the website has a blog, if yes, then scrape titles of last 5 blogs. Can I achieve this in Clay?
Yes, you can use Clay to check if a website has a blog and scrape the titles of the last 5 blog posts. Here's how: 1. Use the "Scrape Website" action in Clay to extract data from the website. 2. In the "Website URL" field, enter the URL of the site you want to check. 3. To find blog posts, you can use the "Links" output field, which returns an array of links from the page. 4. To get blog post titles, use the "Extract Custom Regex" field to create a pattern that matches blog post titles. 5. Process the scraped data to identify blog links and extract the latest 5 titles. Keep in mind that the exact implementation may vary depending on the website's structure. If you need more specific guidance, please share the website URL you're working with.
Mark L.··Hello Zain, that sounds like a task you could take on with "Claygent" – an AI that helps with web research. Good guide on that here: https://www.clay.com/university/lesson/claygent-ai-web-scraper-limitless-research https://www.clay.com/university/lesson/11-ai-prompts-to-automate-prospect-research-with-claygent-automated-outbound Given a website in each row, you could prompt Claygent to scour the site for blog links, and tell it the specific datapoints you want back. When creating a "Use AI" column in clay, I recommend using the "Help Me" button to help transform simple prompts into a more sophisticated, robust one. In addition to the videos above, an illustrated guide on creating Use AI columns: https://www.clay.com/university/guide/use-ai-integration-overview Could you try that, and let me know how it goes?