|
depending on your server, and number of scraper instances you choose to run at the same time, it scraping 100,000+ videos will take at least a good 12 hours and more. espcially with image downloading it will take well over 12 hours.
For those in a hurry, i can provide a pre-scraped mysql database as well. just pop it in and no more waiting to scrape.
you can add a cron job, and let the scraper do its thing, like grabbing the latest videos.
|