![]() |
Robots.txt VS meta robots INDEX
I have a robots text that disables the entire site for indexing
# Make changes for all web spiders User-agent: * Disallow: / but I have robots meta data at the top of each page saying to index it. Which will happen? I don't want it to get indexed but I dont want to disable the plugin adding the robots meta data either. Which one of these rules will trump the other? Thanks in advance - and please don't guess. |
come on guys, I know all the smart people are working Sunday morning. Hook me up with the info please.
|
robots.txt
|
robots.txt
think about it for a sec :) |
they should follow robots.txt first
|
I have no idea
|
robots.txt is looked for first, and followed - so it won't bother hitting the metas if you no-index the site.
Yahoo! will take forever to index and list you if you don't have a robots.txt, for example, even if you have a robots meta tag. |
All times are GMT -7. The time now is 08:21 PM. |
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc123