GoFuckYourself.com - Adult Webmaster Forum - View Single Post

AdultSites · 02-13-2015, 03:46 AM

There is a website that I would need to map, get directory structure, and it is around 1,000,000 pages big. I have no access to the server, or cms. Not all the pages show in Google, and other search engines, and even if they would, it would not be too much help.

I probably need a sitemap, but from what I remember, it is kind of hard to make a sitemap for a big page like that. What would be the best way to work on that / get it done? Something free would be good too. I need website structure, and all urls.

Thanks.

02-13-2015, 03:46 AM
AdultSites Confirmed User Industry Role: Join Date: Feb 2015 Posts: 203	How to map / get idea of a large site? There is a website that I would need to map, get directory structure, and it is around 1,000,000 pages big. I have no access to the server, or cms. Not all the pages show in Google, and other search engines, and even if they would, it would not be too much help. I probably need a sitemap, but from what I remember, it is kind of hard to make a sitemap for a big page like that. What would be the best way to work on that / get it done? Something free would be good too. I need website structure, and all urls. Thanks.