Quote:
Originally Posted by LaoTzu
You probably know this, but you can set a crawl delay in your robots.txt file to slow down googlebot.
|
From what I can tell Google now ignores crawl-delay, and requires webmasters to submit/verify/configure their site via webmaster tools.