|
Some of the bot programs identify themselves with their user agents. You can block based on that.. but really most of them can be set to just identify as Internet explorer.. so not too much you can really *easily* do to stop em if someone wants to download your whole site.
|