Help with robots.txt please
I don't want search engines to index any pages in the root folder and any other folders that has the word page in the url. So for example domain.com/page1.php I don't want indexed.
I believe Disallow: /page will stop any pages starting with page in the root folder, but can a wild card be added to include all files in folders as well?
Thanks for any help.
|