![]() |
Anyone know of a good sitemap detector and scraper?
See title
I need a free or paid tool that a) detects the sitemap url of a site b) scrapes the urls of a site off its sitemap |
just paste mysite.com/sitemap.xml then scrape the good ones. yes i know a scraper...
|
Quote:
|
Quote:
|
Well, the first part should be as easy as going to domain.com/robots.txt and then finding the sitemap. You can then copy/paste it into this site which will extract all the links for you:
https://www.seowl.co/sitemap-extractor/ WG |
Thank you, guys!
WG - thanks for the awesome tool! |
Scrapbox has a plugin
|
Quote:
Scrape all the URL’s from websites using the Sitemap Scraper, included with the ability to do deep crawls. |
Quote:
|
Quote:
|
All times are GMT -7. The time now is 03:02 PM. |
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc123