|
|
|
|
||||
|
Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums. You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today! If you have any problems with the registration process or your account login, please contact us. |
 |
|
|||||||
| Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed. |
|
|
Thread Tools |
08-10-2003, 11:58 AM
|
#1 |
|
ICQ: 175171926
Join Date: Oct 2002
Location: New York, NY
Posts: 11,046
|
google question
although it doesnt index password protected pages, does it even spider them?
is the content counted? or google cant get in either?
__________________
HEY!! Unbeatable hosting! Customer service is top notch! --> ISPrime <-- Do a search on any board, their reputation is rock solid .. for years!! |
|
|
|
08-10-2003, 12:02 PM
|
#2 |
|
So Fucking Banned
Join Date: Oct 2002
Location: chandler, az
Posts: 1,052
|
don't think so.. when i browse google i've never hit a pw protected page before..
|
|
|
|
|
08-10-2003, 12:02 PM
|
#3 |
|
Confirmed User
Join Date: May 2003
Location: The Netherlands
Posts: 2,543
|
If it is protected with .htaccess google can't spider it.
__________________
 
|
|
|
|
|
08-10-2003, 12:05 PM
|
#4 |
|
Confirmed User
Join Date: Jan 2003
Location: Planet Earth
Posts: 1,419
|
Just disallow any pages you don't want spidering in your robots.txt
__________________
|
|
|
|
