Just an FYI for others looking to do stuff like this in PHP.
There's a PHP class available called 'Snoopy', available from here (most recent is 0.94 I believe)
http://sourceforge.net/project/showf...?group_id=2091
Allows you to forge User_Agents, use proxys, request webpages (raw HTML loaded as an array you can scan through) or just request headers to look for 404's and redirects, etc. - all extremely easily.
Great open-source script if you're playing around with stuff like the scripts mentioned in this thread. Cuts a huge chunk of the work out.