|
|
|
|
||||
|
Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums. You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today! If you have any problems with the registration process or your account login, please contact us. |
 |
|
|||||||
| Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed. |
|
|
Thread Tools |
09-18-2008, 07:55 PM
|
#1 |
|
Confirmed User
Industry Role:
Join Date: Mar 2007
Posts: 7,771
|
Do you block cuil.com bot from crawling.
Damn!
I'm thinking to just block this search engine. I've never seen a bot use this much resources. They are crawling every single video page on all my sites at once. This shit will last at least a week maybe two weeks. I have yet to see a single hit of traffic from that engine so WTF?
__________________
  
|
|
|
|
10-11-2008, 03:37 AM
|
#2 |
|
Confirmed User
Join Date: Feb 2007
Location: Sweden
Posts: 5,650
|
It's crawling my sites like crazy right now, but I don't think it's a bad sign. (until the server goes down :P )
__________________
Free 🅑🅘🅣🅒🅞🅘🅝🅢 Every Hour (Yes, really. Free ₿itCoins.) (Signup with ONLY your Email and Password. You can also refer people and get even more.) |
|
|
|
