How to do a batch link check for 404's? - GoFuckYourself.com

08-25-2008, 08:11 PM

Hi guys...
I was just wondering if anybody has a website or software that can take a list of URL's and check for 404's, while making note of which URL's are 404'ing or redirecting. Basically, I'm looking for something like:
http://www.seoconsultants.com/tools/...ch.asp#results

...But it will need to take more than 25 links (hopefully thousands at once), and hopefully only note what URL's are 404'ing or redirecting. Any assistance is appreciated! Thanks!

Example:
http://www.url1.com
http://www.url2.com
http://www.url3.com
http://www.url4.com (maybe this one is a 404 link and redirects, so it gets reported)

Sly · 08-25-2008, 08:14 PM

I think somebody wrote a script here on GFY to do this once... it's very simple, I'm sure one of the programmers could write one up quick.

Jdoughs · 08-25-2008, 09:23 PM

http://home.snafu.de/tilman/xenulink.html

By url, by text file, by page. Pretty much does all error codes I think too, I've had lots of scans hit in the mid hundred thousand, so more then 25 easily.

Zoose · 08-25-2008, 09:58 PM

Quote:

Originally Posted by Jdoughs

http://home.snafu.de/tilman/xenulink.html

By url, by text file, by page. Pretty much does all error codes I think too, I've had lots of scans hit in the mid hundred thousand, so more then 25 easily.

What he said. XENU is the shit

08-25-2008, 11:04 PM

Quote:

Originally Posted by Jdoughs

http://home.snafu.de/tilman/xenulink.html

By url, by text file, by page. Pretty much does all error codes I think too, I've had lots of scans hit in the mid hundred thousand, so more then 25 easily.

This is a great tool, but not exactly what I need. I need to put in a list of URL's and see if they redirect the main page...Not check the links on an already loading page. In essence, this is a TGP/MGP gallery check.

Thanks for the suggestion though! This is helpful as well, once I get past step 1.

ry0t · 08-25-2008, 11:28 PM

My only suggestion is to hire an outsourced programmer on one of the sites to get it done for cheap and fast.

fatfoo · 08-25-2008, 11:58 PM

I agree with ry0t

HomerSimpson · 08-26-2008, 04:11 AM

I can write that kind of script for you very fast...

GrouchyAdmin · 08-26-2008, 04:37 AM

Do you just want it to check for redirects? That's easy.

It accepts one URL per line. If you typo or fuck up, it'll give a generic error. The last one always returns "does not redirect" due to a threading problem with the PHP CGI on my host.

Please don't kill my machine, this simple little thing isn't designed to handle thousands of requests.

08-25-2008, 08:11 PM	#1
shermo Guest Posts: n/a	How to do a batch link check for 404's? Hi guys... I was just wondering if anybody has a website or software that can take a list of URL's and check for 404's, while making note of which URL's are 404'ing or redirecting. Basically, I'm looking for something like: http://www.seoconsultants.com/tools/...ch.asp#results ...But it will need to take more than 25 links (hopefully thousands at once), and hopefully only note what URL's are 404'ing or redirecting. Any assistance is appreciated! Thanks! Example: http://www.url1.com http://www.url2.com http://www.url3.com http://www.url4.com (maybe this one is a 404 link and redirects, so it gets reported)

08-25-2008, 08:14 PM	#2
Sly Let's do some business! Industry Role: Join Date: Sep 2004 Location: Austin, TX Posts: 31,298	I think somebody wrote a script here on GFY to do this once... it's very simple, I'm sure one of the programmers could write one up quick. __________________ Vacares - Web Hosting, Domains, O365, Security & More - Paxum and BTC Accepted Windows VPS now available - 1st month free for GFY members - DM for details Wanted: CCBill pay sites for sale

08-25-2008, 09:23 PM	#3
Jdoughs Confirmed User Industry Role: Join Date: Mar 2004 Location: Great White North Posts: 5,794	http://home.snafu.de/tilman/xenulink.html By url, by text file, by page. Pretty much does all error codes I think too, I've had lots of scans hit in the mid hundred thousand, so more then 25 easily. __________________ LinkSpun - Premier Adult Link Trading Community - ICQ - 464/\281/\250 Be Seen By New Webmasters/Affiliates * Target out webmasters/affiliates based on niches your sites are for less than $20 a month. AmeriNOC - Proudly hosted @ AmeriNOC! Last edited by Jdoughs; 08-25-2008 at 09:25 PM..

08-25-2008, 11:58 PM	#7
fatfoo ICQ:649699063 Industry Role: Join Date: Mar 2003 Posts: 27,763	I agree with ry0t __________________ Send me an email: [email protected]

08-26-2008, 04:11 AM	#8
HomerSimpson Too lazy to set a custom title Industry Role: Join Date: Sep 2005 Location: Springfield Posts: 13,826	I can write that kind of script for you very fast... __________________ Make a bank with Chaturbate - the best selling webcam program Ads that can't be block with AdBlockers !!! /// Best paying popup program (Bitcoin payouts) !!! PHP, MySql, Smarty, CodeIgniter, Laravel, WordPress, NATS... fixing stuff, server migrations & optimizations... My ICQ: 27429884 \| Email:

08-25-2008, 11:28 PM	#6
ry0t So Fucking Banned Industry Role: Join Date: Feb 2003 Location: Boston Posts: 4,160	My only suggestion is to hire an outsourced programmer on one of the sites to get it done for cheap and fast.

08-26-2008, 04:37 AM	#9
GrouchyAdmin Now choke yourself! Industry Role: Join Date: Apr 2006 Posts: 12,085	Do you just want it to check for redirects? That's easy. It accepts one URL per line. If you typo or fuck up, it'll give a generic error. The last one always returns "does not redirect" due to a threading problem with the PHP CGI on my host. Please don't kill my machine, this simple little thing isn't designed to handle thousands of requests. __________________ AIM: GrouchyGfy