GoFuckYourself.com - Adult Webmaster Forum - View Single Post

dunhill · 07-31-2013, 08:53 AM

Better add the noindex metatag on your disallowed files. robots.txt is accessible to everyone and the users can see the disallowed files there.Or block the googlebot user agent from .htaccess

07-31-2013, 08:53 AM
dunhill Confirmed User Industry Role: Join Date: Jul 2013 Posts: 89	Better add the noindex metatag on your disallowed files. robots.txt is accessible to everyone and the users can see the disallowed files there.Or block the googlebot user agent from .htaccess