How do I stop Googlebot from raping my site? 8000+ spider hits?
 

Site is under 5 days old and I get this crap!

Googlebot 3221 49.17 MB 16 Sep 2006 - 04:52
Inktomi Slurp 3045 24.24 MB 16 Sep 2006 - 04:53
Unknown robot (identified by 'crawl') 1610 31.45 MB 16 Sep 2006 - 04:53
MSNBot 59 557.32 KB 16 Sep 2006 - 04:53
AskJeeves 39 574.24 KB 16 Sep 2006 - 04:24


Just kidding. If you want to invest in building some automated traffic targetted to any market email me with your needs and lets chat.

easy to do with .htaccess

that's more than good... congrats! :)

http://www.penguinhosting.net/~tnobo...noshit%201.jpg

its been a long walk

:1orglaugh :1orglaugh :1orglaugh

robots.txt

that's the one

Thanks for all the help! That'll show you google.com

That's where Googlebot went. Send it back my way.

Yahoo Slurp 4890+744 26.77 MB 16 Sep 2006 - 15:00
Googlebot 3265+18 18.58 MB 16 Sep 2006 - 12:07
Alexa (IA Archiver) 430+28 1.68 MB 16 Sep 2006 - 11:41
MSNBot 179+65 964.07 KB 15 Sep 2006 - 02:36
GigaBot 113+67 371.31 KB 10 Sep 2006 - 00:10
Google AdSense 167+1 869.29 KB 16 Sep 2006 - 10:23

What about that Google sitemap thing?

Nice stats but as a note the ones I posted are from a 4.5 day old site that I never have to touch again in my life. ;)

Exactly, which is why I want you to send Googlebot back to me, you botnapper! :1orglaugh

$5000 and I'll hook it up. :upsidedow

You got arachnophobia or something?

good job :thumbsup

25352 1.74% Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.co
43779 3.01% Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/

who needs google

I don't need google. I consider google the same as my woman bringing a sexy girlfriend home for some late night fun. Just an extra bonus that makes my pecker hard.

Congrats I guess....

and you'd tell her to leave?

Not if she knew how to read.

Jesus how 45 megs of html, you must need some sleep right about now.

html or image caching?

im interested in this googlebot deal!

google bot is a whore