GoFuckYourself.com - Adult Webmaster Forum

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)
-   Fucking Around & Business Discussion (https://gfy.com/forumdisplay.php?f=26)
-   -   does Crawl-Delay affect seo for robots.txt (https://gfy.com/showthread.php?t=1048495)

AllAboutCams 12-05-2011 01:43 AM

does Crawl-Delay affect seo for robots.txt
 
does Crawl-Delay affect seo
i.e. if i set my Crawl-Delay for robots.txt
5 seconds
10 seconds
20 seconds
60 seconds
120 seconds

anexsia 12-05-2011 01:46 AM

I'm curious...what would be the benefit of setting a crawl-delay?

AllAboutCams 12-05-2011 01:49 AM

Quote:

Originally Posted by anexsia (Post 18607350)
I'm curious...what would be the benefit of setting a crawl-delay?

apparently that is why my hosting keeps crashing

AllAboutCams 12-05-2011 02:10 AM

google ignores Crawl-delay anyway

anexsia 12-05-2011 02:21 AM

"You can change the crawl rate (the time used by Googlebot to crawl the site) for sites that are at the root level—for example, www.example.com and http://subdomain.example.com. You can't change the crawl rate for sites that are not at the root level—for example, www.example.com/folder.

Note: Crawl rate refers to the speed of Googlebot's requests during the crawl process. It doesn't have any effect on how often we crawl or how deeply we crawl your URL structure."

AdultKing 12-05-2011 03:01 AM

Quote:

Originally Posted by xxxupdate (Post 18607351)
apparently that is why my hosting keeps crashing

If your hosting can't deal with Google crawling then fuck knows how you handle real visitors.

Black All Through 12-05-2011 04:40 AM

Quote:

Originally Posted by AdultKing (Post 18607391)
If your hosting can't deal with Google crawling then fuck knows how you handle real visitors.

:2 cents:

AllAboutCams 12-05-2011 06:09 AM

Quote:

Originally Posted by AdultKing (Post 18607391)
If your hosting can't deal with Google crawling then fuck knows how you handle real visitors.

i have 9 blogs running on Amnoc-r2 with about 1000 hits per day and i keep getting the error can not connect to data base
problem seems to have gone know i went in to web-master tools and set all the crawl rates to the lowest they do
ill monitor the uptime and see what happens
or i could try iinet vps

AdultKing 12-05-2011 07:19 AM

Quote:

Originally Posted by xxxupdate (Post 18607578)
i have 9 blogs running on Amnoc-r2 with about 1000 hits per day and i keep getting the error can not connect to data base
problem seems to have gone know i went in to web-master tools and set all the crawl rates to the lowest they do
ill monitor the uptime and see what happens
or i could try iinet vps

Dude. 1000 hits a day is fuck all.

if by iinet vps you mean iiNet in AU, forget it , you can't host porn anywhere inside AU geographically.

edit: it seems to me you have a configuration or other issue with setup.

BIGTYMER 12-05-2011 07:25 AM

Find a new host.

AllAboutCams 12-05-2011 08:17 AM

Quote:

Originally Posted by AdultKing (Post 18607658)
Dude. 1000 hits a day is fuck all.

if by iinet vps you mean iiNet in AU, forget it , you can't host porn anywhere inside AU geographically.

edit: it seems to me you have a configuration or other issue with setup.

they all work fine know ive never had a problem before could have been the rss feed i had syncing it did add 15,000 post on 3 of the blogs that maybe why i was over using mysql

AdultKing 12-05-2011 06:18 PM

Quote:

Originally Posted by xxxupdate (Post 18607768)
they all work fine know ive never had a problem before could have been the rss feed i had syncing it did add 15,000 post on 3 of the blogs that maybe why i was over using mysql

With all due respect, you really don't have a clue do you ?

CyberHustler 12-05-2011 06:24 PM

Tell them to increase your rams...

AllAboutCams 12-05-2011 06:28 PM

Quote:

Originally Posted by AdultKing (Post 18609353)
With all due respect, you really don't have a clue do you ?

im telling you what happened how can i not have a clue about that


All times are GMT -7. The time now is 11:42 PM.

Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc