When claygent visits a website, can it crawl subpages ?

·

· Sorted by Oldest

Summiya
·
·
No, Claygent currently does not crawl subpages automatically. It only extracts data from the specific URL you provide. If you need to extract data from subpages, try these approaches: Find Subpage URLs First If the main page lists subpage links (e.g., team pages, product pages), you can extract those links using "Extract Links" in Claygent. Then, run Claygent on each subpage URL. If you already have a list of subpage URLs, upload them in a Google Sheet and run Claygent on each row.
David Gabriel (ViewIn Advisory)
·
·
No, but there is a way to extract all data from a website using specific prompting to ensure that all information about an organization is extracted, including their subsidiaries
Josh Burgin
·
·
David Gabriel (ViewIn Advisory) Please explain more.

Summiya
·
·
No, Claygent currently does not crawl subpages automatically. It only extracts data from the specific URL you provide. If you need to extract data from subpages, try these approaches: Find Subpage URLs First If the main page lists subpage links (e.g., team pages, product pages), you can extract those links using "Extract Links" in Claygent. Then, run Claygent on each subpage URL. If you already have a list of subpage URLs, upload them in a Google Sheet and run Claygent on each row.
David Gabriel (ViewIn Advisory)
·
·
No, but there is a way to extract all data from a website using specific prompting to ensure that all information about an organization is extracted, including their subsidiaries
Josh Burgin
·
·
David Gabriel (ViewIn Advisory) Please explain more.