GoFuckYourself.com - Adult Webmaster Forum

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)
-   Webmaster Q & Fuckin' A (https://gfy.com/forumdisplay.php?f=27)
-   -   Web page downloader/scraper...or whatever it's called... (https://gfy.com/showthread.php?t=917613)

FlexxAeon 07-23-2009 04:10 PM

Web page downloader/scraper...or whatever it's called...
 
Seems like there should be some type of application that can do this I just am not sure what the proper term for it would be (never used one). I need to:

- Input several URL's (to FHG's) to batch
- Program will download the pages and the elements it links to (need to at least get the thumbs and the videos)

Anyone know an app that will accomplish this? Or what an app like this would be called? Preferably a freebie as I don't see myself using it very often after this task gets completed

CYF 07-23-2009 04:49 PM

Curl
wget

both are free and will download or mirror a whole website.

cLin 07-23-2009 05:24 PM

On the flip side, how do we stop site ripping? =\ I was thinking javascript but that screws the people who don't use it (which I really shouldn't care about but whatever).

raymor 07-24-2009 10:06 AM

Quote:

Originally Posted by cLin (Post 16099412)
On the flip side, how do we stop site ripping? =\ I was thinking javascript but that screws the people who don't use it (which I really shouldn't care about but whatever).

Strongbox has a couple of anti-ripping features - one that's automatic and one
you can activate. Throttlebox is all about preventing ripping.


All times are GMT -7. The time now is 06:33 PM.

Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc