Caffeine spiders my Mainstream sites in seconds - GoFuckYourself.com

Argos88 · 06-18-2010, 07:00 PM

During the last few months, I noticed that caffeine spiders my Mainstream sites in a matter of seconds... Man, my last post was cached in 10 seconds by google! Then I type the title of the post in Google and there I am! In the first positions with "1 minute ago"! Amazing! And yes, it gets SE traffic if you ask that.

I don't notice the same with my adult sites though, even older adult established sites don't get spidered this fast, like my mainstream sites. Another great reason to switch over to Mainstream.

.

epitome · 06-18-2010, 08:18 PM

I have two social networks and have noticed that the Google bot never leaves. Ever.

rowan · 06-19-2010, 03:25 AM

How could G consistently find new posts within 10 seconds unless it's repeatedly hammering a site? Imagine if all SE bots went fishing like that!

I kind of have the opposite problem, there's too much content for it to ever completely index so it just fetches endlessly, more than 120,000+ per day...

seeandsee · 06-19-2010, 06:16 AM

How much band does google takes that way ??!?

Argos88 · 06-19-2010, 12:17 PM

Quote:

Originally Posted by epitome

I have two social networks and have noticed that the Google bot never leaves. Ever.

What script/software are you using for your social bookmarking sites?

Cyber Fucker · 06-19-2010, 07:22 PM

Google = Evil

and seriously it's really worrying that there is something like Google owned by god knows who and with no serious alternatives...

VHNet · 06-19-2010, 07:26 PM

I'm wondering if google uses its never-ending cookie on people's computers to tell it when new content is viewed and should be spidered?

06-18-2010, 07:00 PM	#1
Argos88 So Fucking Banned Industry Role: Join Date: Sep 2009 Posts: 1,732	Caffeine spiders my Mainstream sites in seconds During the last few months, I noticed that caffeine spiders my Mainstream sites in a matter of seconds... Man, my last post was cached in 10 seconds by google! Then I type the title of the post in Google and there I am! In the first positions with "1 minute ago"! Amazing! And yes, it gets SE traffic if you ask that. I don't notice the same with my adult sites though, even older adult established sites don't get spidered this fast, like my mainstream sites. Another great reason to switch over to Mainstream. .

06-19-2010, 06:16 AM	#4
seeandsee Check SIG! Industry Role: Join Date: Mar 2006 Location: Europe (Skype: gojkoas) Posts: 50,945	How much band does google takes that way ??!? __________________ BUY MY SIG - 50$/Year Contact here

06-19-2010, 07:22 PM	#6
Cyber Fucker Hmm Industry Role: Join Date: Sep 2005 Location: On an endless road around the world for rock and roll. Posts: 12,642	Google = Evil and seriously it's really worrying that there is something like Google owned by god knows who and with no serious alternatives... __________________

06-19-2010, 07:26 PM	#7
VHNet Confirmed User Industry Role: Join Date: May 2005 Location: Chicago Posts: 386	I'm wondering if google uses its never-ending cookie on people's computers to tell it when new content is viewed and should be spidered? __________________ Bringing mainstream marketing to the Adult Industry ICQ: 221976033

06-18-2010, 08:18 PM	#2
epitome So Fucking Lame Industry Role: Join Date: Jun 2009 Location: St. Petersburg, FL Posts: 12,156	I have two social networks and have noticed that the Google bot never leaves. Ever.

06-19-2010, 03:25 AM	#3
rowan Too lazy to set a custom title Join Date: Mar 2002 Location: Australia Posts: 17,393	How could G consistently find new posts within 10 seconds unless it's repeatedly hammering a site? Imagine if all SE bots went fishing like that! I kind of have the opposite problem, there's too much content for it to ever completely index so it just fetches endlessly, more than 120,000+ per day...