Quote:
Originally Posted by zerovic
Also, for some reason.. google will also ignore my robots.txt file here and there.
|
Yeah, seen this happen a few times as well. Even for pages for which it didn't make any sense to get indexed at all, like you've mentioned, outgoing redirect pages and such...
Adding a php header to these type of pages, fixed it for me:
header("X-Robots-Tag: noindex, nofollow", true);