Apache Error Conditional logs -tech help please. - GoFuckYourself.com

camperjohn64 · 04-17-2010, 09:20 AM

Does anyone have experience with the conditional logs of apache?

I have 10x to 20,000 404 errors every day (which are actually correct, and are redirected to affiliate programs), and dont want to log those:
[Sat Apr 17 21:15:25 2010] [error] [client 195.42.102.25] File does not exist: /home/photof/public_html/cats/random.htlm

Is there a way I can get apache to log all errors except those?
http://httpd.apache.org/docs/2.2/logs.html

What environment variable do you think I could use to regex against to acheive this goal?

LoveSandra · 04-17-2010, 10:20 AM

bump bump

Varius · 04-17-2010, 01:56 PM

ErrorLog doesn't support Env conditions, so you'll probably have to use CustomLog and pipe it to a sed command or script to strip out the unwanted lines.

camperjohn64 · 04-17-2010, 02:28 PM

Oh yes I see it is possible to pipe it. Good idea.

I will try this:
ErrorLog "|/usr/bin/php /home/john/test.php"

and see if I can make a little php program that checks for the 404 condition and doesn't write it if so

Or do you think it would run faster if it was written as a shell script?

I don't know I would write a shell script that would "If string not contain "404" and string not contain "not found" then write to /var/log/httpd/error_log"

camperjohn64 · 04-17-2010, 02:50 PM

So far, I don't think I can do this in PHP. I think this is an excersize for another day. But I did find this:

http://www.sudleyplace.com/pipederrorlogs.html

Varius · 04-17-2010, 03:11 PM

I'm not home right now so didn't test this but just try something simple like:

ErrorLog "|/bin/grep -v 'File does not exist:' >> /var/log/httpd/error_log"

Should do the trick.

camperjohn64 · 04-17-2010, 10:26 PM

Hmmm.. I think this is a syntax issue, but it's close

Code:

Starting httpd: grep: does: No such file or directory
grep: not: No such file or directory
grep: exist:: No such file or directory
                                                           [FAILED]

camperjohn64 · 04-17-2010, 10:26 PM

I really think I need to do it in a shell script and figure out how to do it with stdin, just don't know how to yet.

rowan · 04-18-2010, 06:12 AM

Quote:

Originally Posted by camperjohn64

I really think I need to do it in a shell script and figure out how to do it with stdin, just don't know how to yet.

$fp = fopen("php://stdin", "r");
if (!$fp) die("could not open STDIN");

Running a PHP script is pretty clunky, I'd try fixing what Varius gave you. Something like this shoudl work...

ErrorLog "|/bin/grep -v \"File does not exist:\" >> /var/log/httpd/error_log"

camperjohn64 · 04-18-2010, 08:30 AM

Wow that works (with single quotes rather than double quotes):

ErrorLog "|/bin/grep -v \'File does not exist:\' >> /var/log/httpd/error_log"

Now, how do I test for two error conditions with grep?

not found or unable to stat
File does not exist:

I tried this and let's see if it works:

ErrorLog "|/bin/grep -v \'File does not exist:\' |/bin/grep -v \'not found or unable to stat\' >> /var/log/httpd/error_log"

04-17-2010, 09:20 AM	#1
camperjohn64 Confirmed User Industry Role: Join Date: Feb 2005 Location: Los Angeles Posts: 1,531	Apache Error Conditional logs -tech help please. Does anyone have experience with the conditional logs of apache? I have 10x to 20,000 404 errors every day (which are actually correct, and are redirected to affiliate programs), and dont want to log those: [Sat Apr 17 21:15:25 2010] [error] [client 195.42.102.25] File does not exist: /home/photof/public_html/cats/random.htlm Is there a way I can get apache to log all errors except those? http://httpd.apache.org/docs/2.2/logs.html What environment variable do you think I could use to regex against to acheive this goal? __________________ www.gimmiegirlproductions.com

04-17-2010, 01:56 PM	#3
Varius Confirmed User Industry Role: Join Date: Jun 2004 Location: New York, NY Posts: 6,890	ErrorLog doesn't support Env conditions, so you'll probably have to use CustomLog and pipe it to a sed command or script to strip out the unwanted lines. __________________ Skype variuscr - Email varius AT gmail

04-17-2010, 02:28 PM	#4
camperjohn64 Confirmed User Industry Role: Join Date: Feb 2005 Location: Los Angeles Posts: 1,531	Oh yes I see it is possible to pipe it. Good idea. I will try this: ErrorLog "\|/usr/bin/php /home/john/test.php" and see if I can make a little php program that checks for the 404 condition and doesn't write it if so Or do you think it would run faster if it was written as a shell script? I don't know I would write a shell script that would "If string not contain "404" and string not contain "not found" then write to /var/log/httpd/error_log" __________________ www.gimmiegirlproductions.com

04-17-2010, 02:50 PM	#5
camperjohn64 Confirmed User Industry Role: Join Date: Feb 2005 Location: Los Angeles Posts: 1,531	So far, I don't think I can do this in PHP. I think this is an excersize for another day. But I did find this: http://www.sudleyplace.com/pipederrorlogs.html __________________ www.gimmiegirlproductions.com

04-17-2010, 03:11 PM	#6
Varius Confirmed User Industry Role: Join Date: Jun 2004 Location: New York, NY Posts: 6,890	I'm not home right now so didn't test this but just try something simple like: ErrorLog "\|/bin/grep -v 'File does not exist:' >> /var/log/httpd/error_log" Should do the trick. __________________ Skype variuscr - Email varius AT gmail

04-17-2010, 10:20 AM	#2
LoveSandra So Fucking Banned Join Date: Aug 2008 Location: Just Blow Me Posts: 10,551	bump bump

04-17-2010, 10:26 PM	#7
camperjohn64 Confirmed User Industry Role: Join Date: Feb 2005 Location: Los Angeles Posts: 1,531	Hmmm.. I think this is a syntax issue, but it's close Code: Starting httpd: grep: does: No such file or directory grep: not: No such file or directory grep: exist:: No such file or directory [FAILED] __________________ www.gimmiegirlproductions.com

04-18-2010, 08:30 AM	#10
camperjohn64 Confirmed User Industry Role: Join Date: Feb 2005 Location: Los Angeles Posts: 1,531	Wow that works (with single quotes rather than double quotes): ErrorLog "\|/bin/grep -v \'File does not exist:\' >> /var/log/httpd/error_log" Now, how do I test for two error conditions with grep? not found or unable to stat File does not exist: I tried this and let's see if it works: ErrorLog "\|/bin/grep -v \'File does not exist:\' \|/bin/grep -v \'not found or unable to stat\' >> /var/log/httpd/error_log" __________________ www.gimmiegirlproductions.com