GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Googlebot jerks me 100,000 times a day but Google won't show me more love

rowan · 08-05-2009, 04:14 PM

I just thought of a sticky tape and bandaids solution to the crawl-delay issue... since Googlebot appears to fetch sequentially, why not just add a delay yourself?

if (strpos($_SERVER["HTTP_USER_AGENT"], " Googlebot/") > 0) sleep(2);

08-05-2009, 04:14 PM
rowan Too lazy to set a custom title Join Date: Mar 2002 Location: Australia Posts: 17,393	I just thought of a sticky tape and bandaids solution to the crawl-delay issue... since Googlebot appears to fetch sequentially, why not just add a delay yourself? if (strpos($_SERVER["HTTP_USER_AGENT"], " Googlebot/") > 0) sleep(2);