Thread: does Crawl-Delay affect seo for robots.txt
View Single Post
Old 12-05-2011, 03:01 AM  
AdultKing
Raise Your Weapon
 
AdultKing's Avatar
 
Industry Role:
Join Date: Jun 2003
Location: Outback Australia
Posts: 15,605
Quote:
Originally Posted by xxxupdate View Post
apparently that is why my hosting keeps crashing
If your hosting can't deal with Google crawling then fuck knows how you handle real visitors.
AdultKing is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote