![]() |
Do you block cuil.com bot from crawling.
Damn!
I'm thinking to just block this search engine. I've never seen a bot use this much resources. They are crawling every single video page on all my sites at once. This shit will last at least a week maybe two weeks. I have yet to see a single hit of traffic from that engine so WTF? |
It's crawling my sites like crazy right now, but I don't think it's a bad sign. (until the server goes down :P )
|
All times are GMT -7. The time now is 01:38 PM. |
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc123