GoFuckYourself.com - Adult Webmaster Forum - View Single Post

Dawgy · 10-28-2003, 01:52 PM

? i dont get it. haha

isnt robots.txt supposed to disallow files that actually exist? thus the reason for disallowing them to be spidered?

i dont know my SE stuff... so... yeah im stupid :P

10-28-2003, 01:52 PM
Dawgy Confirmed User Join Date: Jul 2001 Posts: 5,856	? i dont get it. haha isnt robots.txt supposed to disallow files that actually exist? thus the reason for disallowing them to be spidered? i dont know my SE stuff... so... yeah im stupid :P __________________ the revolution is coming.