![]() |
does Crawl-Delay affect seo for robots.txt
does Crawl-Delay affect seo
i.e. if i set my Crawl-Delay for robots.txt 5 seconds 10 seconds 20 seconds 60 seconds 120 seconds |
I'm curious...what would be the benefit of setting a crawl-delay?
|
Quote:
|
google ignores Crawl-delay anyway
|
"You can change the crawl rate (the time used by Googlebot to crawl the site) for sites that are at the root level—for example, www.example.com and http://subdomain.example.com. You can't change the crawl rate for sites that are not at the root level—for example, www.example.com/folder.
Note: Crawl rate refers to the speed of Googlebot's requests during the crawl process. It doesn't have any effect on how often we crawl or how deeply we crawl your URL structure." |
Quote:
|
Quote:
|
Quote:
problem seems to have gone know i went in to web-master tools and set all the crawl rates to the lowest they do ill monitor the uptime and see what happens or i could try iinet vps |
Quote:
if by iinet vps you mean iiNet in AU, forget it , you can't host porn anywhere inside AU geographically. edit: it seems to me you have a configuration or other issue with setup. |
Find a new host.
|
Quote:
|
Quote:
|
Tell them to increase your rams...
|
Quote:
|
All times are GMT -7. The time now is 11:42 PM. |
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc