GoFuckYourself.com - Adult Webmaster Forum - Tech How to index AVS tube? (robots.txt issue)

- Webmaster Q & Fuckin' A (https://gfy.com/forumdisplay.php?f=27)

- - Tech How to index AVS tube? (robots.txt issue) (https://gfy.com/showthread.php?t=1184803)

How to index AVS tube? (robots.txt issue)

Hi guys,

I added a tube (with AVS script) in Google Webmaster Tools, but the issue is with the robots.txt file which is blocking everything.

Here is my AVS tube robots.txt:

Code:

User-agent: *

Disallow: /siteadmin/

Disallow: /cache/

Disallow: /include/

Disallow: /install/

Disallow: /language/

Disallow: /media/

Disallow: /tmp/

Disallow: /classes/

Disallow: /templates/

Disallow: /scripts/

Disallow: /addons/

Disallow: /cron/

Disallow: /contrib/

Any idea how to fix this and get the tube indexed?

Thank you!

Thanks for the reply.

I've added:

Code:

Allow: /

And didn't change anything else in the robots.txt, only added "Allow: /"

I still get errors in Webmaster Tools.

Code:

Warnings 



Url blocked by robots.txt.



Sitemap contains urls which are blocked by robots.txt.



Issues: 272



Value: http://www.domain.com/media/videos/tmb/1/1.jpg

-

Feb 17, 2016

Value: http://www.domain.com/media/videos/tmb/10/1.jpg

-

Feb 17, 2016

Value: http://www.domain.com/media/videos/tmb/100/1.jpg

-

Feb 17, 2016

Now, I have to mention I've added both versions of site, www. and non-www.