GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Do you let google crawler see your members area?

DWB · 02-20-2013, 10:10 AM

Google used to pay attention to robots.txt but seems it only ignores mine now and does what it wants.

Dig your site btw.

02-20-2013, 10:10 AM
DWB Registered User Industry Role: Join Date: Jul 2003 Location: Encrypted. Access denied. Posts: 31,779	Google used to pay attention to robots.txt but seems it only ignores mine now and does what it wants. Dig your site btw.