GoFuckYourself.com - Adult Webmaster Forum

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)
-   Fucking Around & Business Discussion (https://gfy.com/forumdisplay.php?f=26)
-   -   Lots of unwanted pages in site:adultsitesurfer.com in google/yahoo, help! (https://gfy.com/showthread.php?t=511289)

Socks 09-01-2005 04:33 PM

Lots of unwanted pages in site:adultsitesurfer.com in google/yahoo, help!
 
Hey, having some problems with pages appearing in both google and yahoo that we don't need indexed. We've tried to use robots.txt but it doesn't seem to be working.

http://www.adultsitesurfer.com/robots.txt

The unwanted links are like http://www.adultsitesurfer.com/comme...p?site_id=1306

and http://www.adultsitesurfer.com/email...hp?site_id=410

Yahoo page of bad links

Not sure what to do? Any help appreciated!

Machete_ 09-01-2005 04:42 PM

read this
http://www.robotstxt.org/wc/meta-user.html

PSD 09-01-2005 04:42 PM

stick those two files (comment and email review) in a sub directory and then add the subdirectory to your robots file. I think they will eventually be removed if you do that but if you don't want to wait at Google you can remove them at:

http://services.google.com/urlconsole/controller

Machete_ 09-01-2005 04:42 PM

<meta name="robots" content="noindex,nofollow">

put that in the header, and you are 95% sure it wont index it

BastarD 09-01-2005 04:44 PM

robots.txt would do all the trick.


All times are GMT -7. The time now is 05:17 PM.

Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc