GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Does anyone even bother with robots.txt anymore?

Porndex · 01-24-2013, 07:58 AM

I use it to block stuff that wouldn't be worth indexing. Works a treat with the major engines.

01-24-2013, 07:58 AM
Porndex Registered User Industry Role: Join Date: Oct 2012 Location: Germany Posts: 28	I use it to block stuff that wouldn't be worth indexing. Works a treat with the major engines.