Crawling Legality Issues
 

Hello all,

I was wondering how crawling is looked upon. Me and my business partner want to crawl some tube sites (so we can create our own database structure instead of using an existing one).

The only information we are after are the embed codes and meta information such as tags and duration etc.

Is this okay or frowned upon? ;)

Thanks in advance,

James

just do it
why would you ask ?

Because webmasters could easily block us. Maybe it's nice to ask first or whatever. So that's why I'm asking if it's an issue.

Welcome to GFY :thumbsup and good luck

It should be ok. But do not copy titles and tags 1:1 to your website. Make it a little different. Duplicity is bad.

James... the term is "scraping".

Crawling is simply going from link to link finding web pages. Scraping is collecting information with the intent to republish.

Quora: What is the difference between crawling and scraping?