Quote:
Originally Posted by d-null
I robots.txt disallow archive.org on all my sites  |
and if anyone is wondering how to do it, here is the info:
http://www.archive.org/about/exclude.php
basically, all you need to do is add this to your robots.txt file, and it will also remove all records of your site at archive.org:
Code:
User-agent: ia_archiver
Disallow: /