|
|
|
|
||||
|
Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums. You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today! If you have any problems with the registration process or your account login, please contact us. |
 |
|
|||||||
| Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed. |
|
|
Thread Tools |
05-08-2005, 10:13 AM
|
#1 |
|
Guest
Posts: n/a
|
robots.txt question
If I have a whole bunch of pages of type domain.com/script.php?id=XXX indexed in SEs, and I would like to block spiders from indexing them, do I use
User-agent: * Disallow: script.php in order to block all the pages? Thanks |
|
|
05-08-2005, 10:14 AM
|
#2 |
|
[----------------------]
Join Date: Aug 2001
Posts: 14,486
|
yep, correct
disallow: /script.php don't forget the / |
|
|
|
05-08-2005, 10:22 AM
|
#3 |
|
Guest
Posts: n/a
|
thanks for the info Trax
|
|
|
|
05-08-2005, 01:31 PM
|
#4 |
|
Confirmed User
Join Date: Apr 2005
Posts: 254
|
Thanks for the info.
This is something that i was wondering too. 
__________________
|
|
|
|
|
05-08-2005, 03:03 PM
|
#5 |
|
[----------------------]
Join Date: Aug 2001
Posts: 14,486
|
youre welcome
|
|
|
|
