Quote:
Originally Posted by TurboB
Google respects robots.txt rules. If the bot does not crawl the site - it is impossible to get indexed.
|
This is absolute nonsense! yes google respects robot txt. but disallow does not behave the same as no index
google themselves say there is a difference between noindex and disallow
and
Of course setting up a whitelbel with the goal of ranking it in google is a waste of time.
But that doesn't mean there aren't people who get converting google hits on their whitelabel. Due to keyword matching domain names, and in our case tag pages temporarily getting indexed due to backlinks