I would recommend talking with a programmer and having them create a customized scraper for you that does exactly what you want, instead of trying to piece together a bunch of different software packages. It will cost you a little money (although not much) but will save you a crazy amount of time.
Automate as much as possible.
|