GoFuckYourself.com - Adult Webmaster Forum - View Single Post - BLOCKING crawlers whose SE's send you zero traffic

rowan · 10-20-2009, 06:57 PM

LOL. Even though the SE bots are actually the majority of the "traffic" loading the site this issue is nothing to do with bandwidth. It is more multiple database accesses and overall load of the server.

10-20-2009, 06:57 PM
rowan Too lazy to set a custom title Join Date: Mar 2002 Location: Australia Posts: 17,393	LOL. Even though the SE bots are actually the majority of the "traffic" loading the site this issue is nothing to do with bandwidth. It is more multiple database accesses and overall load of the server.