View Single Post
Old 05-20-2012, 02:13 PM  
Darrell
Confirmed User
 
Join Date: Feb 2003
Location: In bed asleep
Posts: 803
Help with robots.txt please

I don't want search engines to index any pages in the root folder and any other folders that has the word page in the url. So for example domain.com/page1.php I don't want indexed.

I believe Disallow: /page will stop any pages starting with page in the root folder, but can a wild card be added to include all files in folders as well?

Thanks for any help.
Darrell is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote