help with htaccess - GoFuckYourself.com

beemk · 05-30-2002, 04:51 PM

i tried this htaccess and it send all my traffic to my sponsor, can someone tell me whats wrong?

AuthUserFile /dev/null
AuthGroupFile /dev/null
AddType text/html .html .htm

<Limit GET>
order allow,deny
allow from all
deny from 212.138.47.*
deny from 194.84.65.*
deny from 212.113.37.194
deny from 217.80.182.65
deny from 213.76.163.16
allow from all
</Limit>

RewriteEngine On

RewriteCond %{HTTP_REFERER} www.quality-thumbs.com [OR]
RewriteCond %{HTTP_REFERER} quality-thumbs.com [or]

RewriteRule /* http://www.lightspeedcash.com/link.php?s=LSU&r=245496 [L]

RewriteOptions inherit
RewriteCond %{HTTP_USER_AGENT} ^DISCo\Pump.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Drip.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Gets.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^IBrowse.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\Ninja.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^JustView.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\tool.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\PiX.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\Explorer.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\Foto.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Pockey.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Slurp.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^SpaceBison.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\Pro.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\Image\Collector.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\Sucker.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Webster.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^ia_archiver.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^lftp.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut.*
RewriteCond %{HTTP_REFERER} ^http://.*trianca.com/*$ [NC]
RewriteCond %{HTTP_REFERER} ^http://.*66.40.119.4:8084/*$ [NC]
RewriteCond %{HTTP_REFERER} ^http://.*chatropolis.com/*$ [NC]
RewriteRule /* http://www.lightspeedcash.com/link.php?s=LSU&r=245496 [R,L]
#RewriteCond %{HTTP_REFERER} !^http://130.94.142.130/.*$ [NC]
#RewriteCond %{HTTP_REFERER} !^http://.*idthumbs.com/.*$ [NC]
RewriteRule .*[Jj][Pp][Gg]$|.*[Gg][Ii][Ff]$ http://www.idthumbs.com/

ErrorDocument 403 /index.shtml
ErrorDocument 404 /index.shtml

fiveyes · 05-31-2002, 01:10 PM

Quote:

Originally posted by beemk
i tried this htaccess and it send all my traffic to my sponsor, can someone tell me whats wrong?

No one else seems willing to give it a shot, so ...

I see a number of errors and problems within the code, which apparently was pasted together from different examples that were found to address different issues. Nothing really wrong with that, except you inherent the typos and possible ignorance of the original author of each snippet. That, and modifying the code to suit your specifics without a grasp of the underlying concepts and rules of the grammar almost assures failure.
Anyway:

Code:

AuthUserFile /dev/null
AuthGroupFile /dev/null
AddType text/html .html .htm

All right, except I don't see the need for the AddType, that should be taken care of in the server's config files.

Code:

&lt;Limit GET>
order allow,deny
allow from all
deny from 212.138.47.*
deny from 194.84.65.*
deny from 212.113.37.194
deny from 217.80.182.65
deny from 213.76.163.16
allow from all
&lt;Limit>

Remove the line in red, it's a repeat.

Code:

RewriteEngine On

RewriteCond %{HTTP_REFERER} www.quality-thumbs.com [OR]
RewriteCond %{HTTP_REFERER} quality-thumbs.com [or]

RewriteRule /* http://www.lightspeedcash.com/link.php?s=LSU&r=245496 [L]

RewriteOptions inherit

This is the error that is causing everyone to be directed to your sponsor- you have a dangling OR flag without a following condition. A few other problems are in this section of code as well, such as not anticipating a capitalized referrer URL. Try using this instead:

Code:

RewriteEngine On
RewriteOptions inherit

# This next condition takes care of www.quality-thumbs.com
# as well as QUALITY-THUMBS.COM, sending their referrals to your sponsor
RewriteCond %{HTTP_REFERER} quality-thumbs.com [NC]
RewriteRule /* http://www.lightspeedcash.com/link.php?s=LSU&r=245496 [L]

Code:

RewriteCond %{HTTP_USER_AGENT} ^DISCo\Pump.* [OR]
# ... [SNIP] ...
RewriteCond %{HTTP_USER_AGENT} ^lftp.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut.* 
RewriteCond %{HTTP_REFERER} ^http://.*trianca.com/*$ [NC]
RewriteCond %{HTTP_REFERER} ^http://.*66.40.119.4:8084/*$ [NC]
RewriteCond %{HTTP_REFERER} ^http://.*chatropolis.com/*$ [NC]
RewriteRule /* http://www.lightspeedcash.com/link.php?s=LSU&r=245496 [R,L]

OUCH!!! This section of code ends up doing nothing but slow your server down!
Keep in mind that Conditions are ANDed together by default. So, this section is saying- if any of the User_Agents are robots PLUS they are referred from trianca.com TOGETHER WITH trianca.com AND 66.40.119.4:8084 AS WELL AS chatropolis.com. Well, since there can be only one referrer for each request, as soon as you require two different ones in the Conditions, nothing will match.

Now, you already have a section for quality-thumbs.com referrals to go to LightSpeed, so let's re-do that to include the others as well:

Code:

RewriteEngine On
RewriteOptions inherit

RewriteCond %{HTTP_REFERER} quality-thumbs.com [NC,OR]
RewriteCond %{HTTP_REFERER} trianca.com [NC,OR]
RewriteCond %{HTTP_REFERER} 66.40.119.4:8084 [OR]
RewriteCond %{HTTP_REFERER} chatropolis.com [NC]
RewriteRule /* http://www.lightspeedcash.com/link.php?s=LSU&r=245496 [L]

But, what about that list of robots? Are you sure you want to pass even one of them onto your sponsor?!? I doubt that would be advisable, better to just refuse them outright, like so:

Code:

RewriteCond %{HTTP_USER_AGENT} ^DISCo\Pump.* [OR]
# ... [SNIP] ...
RewriteCond %{HTTP_USER_AGENT} ^lftp.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut.*
RewriteRule .* - [F,L]

But, look at the RewriteRule on this bit- see that ".*"? That is actually a condition, nothing is going to be rewritten unless what is in this position matches the request. In fact, it is the FIRST condition that is checked each time! That's right, before any of the preceding RewriteCond's are considered, the rewrite engine first checks to see if there's a reason to consider them- it looks at the RewriteRule to see if it matches the request and ONLY IF IT DOES, then goes through the list of the RewriteCond's before it.
Now, the point is, ".*" is a wildcard match, it's going to match any and all requests. So, the rewrite engine is going to go through that list above it for every single hit. That is, unless a RewriteRule before it has an "L" flag, where the rule is never encountered. So, this code bit should go at the very last. No sense in making your server process all of this if the next rule sends the visitor off to your sponsor...

Code:

#RewriteCond %{HTTP_REFERER} !^http://130.94.142.130/.*$ [NC] 
#RewriteCond %{HTTP_REFERER} !^http://.*idthumbs.com/.*$ [NC] 
RewriteRule .*[Jj][Pp][Gg]$|.*[Gg][Ii][Ff]$ http://www.idthumbs.com/

Those two #'s are causing the following Conditions to be commented out, which results in the RewriteRule to stand alone. In other words, ALL of your image requests are going to be changed, not a good thing! Also, you're not accounting for those browser's that don't supply a referrer. And, keep in mind that a rewrite to http://www.idthumbs.com/ is going to be stripped out since it's yourself, unless you add a "R" flag for an external redirect. So:

Code:

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://130.94.142.130 [NC] 
RewriteCond %{HTTP_REFERER} !^http://idthumbs.com/ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.idthumbs.com/ [NC]

RewriteRule .*[Jj][Pp][Gg]$|.*[Gg][Ii][Ff]$ /

Code:

ErrorDocument 403 /index.shtml
ErrorDocument 404 /index.shtml

Good enough, but this usually goes to top of the .htaccess file by convention.

So, the final result (which should give you the desired behavior) would look like this:

Code:

AuthUserFile /dev/null
AuthGroupFile /dev/null

# Remove the "#" from the next line only if needed
#AddType text/html .html .htm

ErrorDocument 403 /index.shtml
ErrorDocument 404 /index.shtml

&lt;Limit GET>
order allow,deny
allow from all
deny from 212.138.47.*
deny from 194.84.65.*
deny from 212.113.37.194
deny from 217.80.182.65
deny from 213.76.163.16
&lt;/Limit>

RewriteEngine On
RewriteOptions inherit

RewriteCond %{HTTP_REFERER} quality-thumbs.com [NC,OR]
RewriteCond %{HTTP_REFERER} trianca.com [NC,OR]
RewriteCond %{HTTP_REFERER} 66.40.119.4:8084 [OR]
RewriteCond %{HTTP_REFERER} chatropolis.com [NC]
RewriteRule .* http://www.lightspeedcash.com/link.php?s=LSU&r=245496 [L]

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://130.94.142.130 [NC] 
RewriteCond %{HTTP_REFERER} !^http://idthumbs.com/ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.idthumbs.com/ [NC]
RewriteRule .*[Jj][Pp][Gg]$|.*[Gg][Ii][Ff]$ / [L]

RewriteCond %{HTTP_USER_AGENT} ^DISCo\Pump.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Drip.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^FlashGet.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^GetRight.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Gets.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Grafula.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^IBrowse.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^InterGET.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Internet\Ninja.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^JetCar.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^JustView.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^MIDown\tool.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Mister\PiX.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^NearSite.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^NetSpider.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Offline\Explorer.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Papa\Foto.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Pockey.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^ReGet.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Slurp.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^SpaceBison.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Teleport.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Teleport\Pro.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^WebAuto.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^WebCopier.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^WebFetch.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^WebReaper.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^WebSauger.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^WebStripper.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^WebZIP.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Web\Image\Collector.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Web\Sucker.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Webster.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^Wget.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^eCatch.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^ia_archiver.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^lftp.* [OR] 
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut.*
RewriteRule .* - [F]

Anyway, that's my take on it. If it doesn't do exactly what you want, I hope it at least steers you in the right direction...

salsbury · 05-31-2002, 01:20 PM

i'm not a fan of long .htaccess files personally (i've seen them drive up the server CPU to 0% idle in nothing flat), nor am i a big fan of blocking based on agent (how long til all websuckers default to IE's agent header)*, but i wonder if it'd just be better to run a whitelist of agents. i'd bet it'd be a lot smaller, especially since you can include 'Mozilla' to cover 95% of all browsers.

fiveyes · 05-31-2002, 01:32 PM

Good point, salsbury! I'd overlooked that while wading through everthing else. Plus, the User Agent can be faked as well.

Perhaps the best way to deal with this issue would be along the lines of a hitbot detector. Anyone know if there's such a program around that nails these web vacuums?

salsbury · 05-31-2002, 01:39 PM

i've set up scripts (real basic) that check for patterns (regular expressions) in the last say 5000 lines of a log file. if they reach a certain threshold it blocks them for a period, say an hour. harsh, but it catches 'em. the thing about it though, it's requires a lot of planning to set it up right - you want to be sure not to block someone for loading your page if it has 50 images. for example on one server i have it set to block if you've hit 25 different CGIs (other than a traffic counting one and admin ones, gotta exclude some too) in 5 minutes or so. this was necessary because the suckers were coming in and killing a server with their simultaneous connections - denial of service attack, basically. but this might not be appropriate for some sites, in fact probably not for most sites.

i think the only way to do it really well is to have something sitting in front of apache proxying traffic that handles the throttling based on whatever it determines is a "session" ip address, cookie, URL prepend, whatever. doing it in apache is just doomed of course - you have a limited number of httpd children ('cause of memory) and if some spider comes along and eats up 100 connections (it has happened), and they're all "sleeping" for a period while it throttles them, that's 100 connections that can't be used for anyone else. lame.

beemk · 05-31-2002, 02:33 PM

thanks a ton for taking the time to reply, i really appreciate it.

beemk · 05-31-2002, 02:40 PM

i just tested it and it worked except the error 404. i tried this the other day with just the error404 line and it still didnt work. any ideas?

fiveyes · 05-31-2002, 05:34 PM

Quote:

Originally posted by beemk
i just tested it and it worked except the error 404. i tried this the other day with just the error404 line and it still didnt work. any ideas?

Apparently the AllowOveride option called FileInfo is not set in the server's config file. Ask your tech admin to set it. In fact, ask him to set "AllowOveride All", which is the least restrictive.

Hope that helps!

beemk · 05-31-2002, 07:09 PM

thanks once again 5eyes

Choker · 05-31-2002, 07:28 PM

fiveyes, you seem to be the authority on htaccess. Why don't you charge like 25$ each to make htaccess files for guys. This is a much needed market. I get people asking me all the time, and I have NO FUCKING clue. I just get friends to help me, and I hate asking anyone for something without paying them for it. If interested, let me know as I sell a lot of traffic to newbies, and they ask me all the time about this. Lots of business for this. Htaccess is a pain in the ass for oldtimers as well as newbies. Someone just needs to do it smart.

05-30-2002, 04:51 PM	#1
beemk CLICK HERE Industry Role: Join Date: Jan 2002 Posts: 20,829	help with htaccess i tried this htaccess and it send all my traffic to my sponsor, can someone tell me whats wrong? AuthUserFile /dev/null AuthGroupFile /dev/null AddType text/html .html .htm <Limit GET> order allow,deny allow from all deny from 212.138.47.* deny from 194.84.65.* deny from 212.113.37.194 deny from 217.80.182.65 deny from 213.76.163.16 allow from all </Limit> RewriteEngine On RewriteCond %{HTTP_REFERER} www.quality-thumbs.com [OR] RewriteCond %{HTTP_REFERER} quality-thumbs.com [or] RewriteRule /* http://www.lightspeedcash.com/link.php?s=LSU&r=245496 [L] RewriteOptions inherit RewriteCond %{HTTP_USER_AGENT} ^DISCo\Pump.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Drip.* [OR] RewriteCond %{HTTP_USER_AGENT} ^EirGrabber.* [OR] RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro.* [OR] RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE.* [OR] RewriteCond %{HTTP_USER_AGENT} ^FlashGet.* [OR] RewriteCond %{HTTP_USER_AGENT} ^GetRight.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Gets.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Grafula.* [OR] RewriteCond %{HTTP_USER_AGENT} ^IBrowse.* [OR] RewriteCond %{HTTP_USER_AGENT} ^InterGET.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Internet\Ninja.* [OR] RewriteCond %{HTTP_USER_AGENT} ^JetCar.* [OR] RewriteCond %{HTTP_USER_AGENT} ^JustView.* [OR] RewriteCond %{HTTP_USER_AGENT} ^MIDown\tool.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Mister\PiX.* [OR] RewriteCond %{HTTP_USER_AGENT} ^NearSite.* [OR] RewriteCond %{HTTP_USER_AGENT} ^NetSpider.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\Explorer.* [OR] RewriteCond %{HTTP_USER_AGENT} ^PageGrabber.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Papa\Foto.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Pockey.* [OR] RewriteCond %{HTTP_USER_AGENT} ^ReGet.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Slurp.* [OR] RewriteCond %{HTTP_USER_AGENT} ^SpaceBison.* [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Teleport.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Teleport\Pro.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebAuto.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebCopier.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebFetch.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebReaper.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebStripper.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebWhacker.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebZIP.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\Image\Collector.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\Sucker.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Webster.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Wget.* [OR] RewriteCond %{HTTP_USER_AGENT} ^eCatch.* [OR] RewriteCond %{HTTP_USER_AGENT} ^ia_archiver.* [OR] RewriteCond %{HTTP_USER_AGENT} ^lftp.* [OR] RewriteCond %{HTTP_USER_AGENT} ^tAkeOut.* RewriteCond %{HTTP_REFERER} ^http://.trianca.com/$ [NC] RewriteCond %{HTTP_REFERER} ^http://.66.40.119.4:8084/$ [NC] RewriteCond %{HTTP_REFERER} ^http://.chatropolis.com/$ [NC] RewriteRule /* http://www.lightspeedcash.com/link.php?s=LSU&r=245496 [R,L] #RewriteCond %{HTTP_REFERER} !^http://130.94.142.130/.$ [NC] #RewriteCond %{HTTP_REFERER} !^http://.idthumbs.com/.$ [NC] RewriteRule .[Jj][Pp][Gg]$\|.*[Gg][Ii][Ff]$ http://www.idthumbs.com/ ErrorDocument 403 /index.shtml ErrorDocument 404 /index.shtml

05-31-2002, 01:20 PM	#3
salsbury Confirmed User Join Date: Feb 2002 Location: Seattle Posts: 1,070	i'm not a fan of long .htaccess files personally (i've seen them drive up the server CPU to 0% idle in nothing flat), nor am i a big fan of blocking based on agent (how long til all websuckers default to IE's agent header)*, but i wonder if it'd just be better to run a whitelist of agents. i'd bet it'd be a lot smaller, especially since you can include 'Mozilla' to cover 95% of all browsers. __________________

05-31-2002, 01:32 PM	#4
fiveyes Confirmed User Join Date: Aug 2001 Location: New Orleans Posts: 1,680	Good point, salsbury! I'd overlooked that while wading through everthing else. Plus, the User Agent can be faked as well. Perhaps the best way to deal with this issue would be along the lines of a hitbot detector. Anyone know if there's such a program around that nails these web vacuums? __________________ <CENTER><A HREF="http://www.hot-off-bourbon.com/" target="_blank"><IMG SRC="http://www.hot-off-bourbon.com/images/hob-logosmall.jpg" border="0"></A> <FONT face="Comic Sans MS" SIZE="-1"><I>Mardi Gras, Spring Break, Wet-T, Night Club Action, UpSkirt, Oil Wrestling, Voyeur</I></FONT></CENTER>

05-31-2002, 01:39 PM	#5
salsbury Confirmed User Join Date: Feb 2002 Location: Seattle Posts: 1,070	i've set up scripts (real basic) that check for patterns (regular expressions) in the last say 5000 lines of a log file. if they reach a certain threshold it blocks them for a period, say an hour. harsh, but it catches 'em. the thing about it though, it's requires a lot of planning to set it up right - you want to be sure not to block someone for loading your page if it has 50 images. for example on one server i have it set to block if you've hit 25 different CGIs (other than a traffic counting one and admin ones, gotta exclude some too) in 5 minutes or so. this was necessary because the suckers were coming in and killing a server with their simultaneous connections - denial of service attack, basically. but this might not be appropriate for some sites, in fact probably not for most sites. i think the only way to do it really well is to have something sitting in front of apache proxying traffic that handles the throttling based on whatever it determines is a "session" ip address, cookie, URL prepend, whatever. doing it in apache is just doomed of course - you have a limited number of httpd children ('cause of memory) and if some spider comes along and eats up 100 connections (it has happened), and they're all "sleeping" for a period while it throttles them, that's 100 connections that can't be used for anyone else. lame. __________________

05-31-2002, 07:28 PM	#10
Choker Confirmed User Industry Role: Join Date: Apr 2001 Location: Orlando Posts: 9,024	fiveyes, you seem to be the authority on htaccess. Why don't you charge like 25$ each to make htaccess files for guys. This is a much needed market. I get people asking me all the time, and I have NO FUCKING clue. I just get friends to help me, and I hate asking anyone for something without paying them for it. If interested, let me know as I sell a lot of traffic to newbies, and they ask me all the time about this. Lots of business for this. Htaccess is a pain in the ass for oldtimers as well as newbies. Someone just needs to do it smart. __________________ ICQ me lets make a deal 116894466 Need dating, cam, or tube traffic? I got it.http://http://www.chokertraffic.com The Original http://www.chokertraffic.com/ Premium country pop-unders from $1.50 per k. I challenge you to compare this traffic to any other brokers. http://www.chokertraffic.com/public/tabs.php?t=o

05-31-2002, 02:33 PM	#6
beemk CLICK HERE Industry Role: Join Date: Jan 2002 Posts: 20,829	thanks a ton for taking the time to reply, i really appreciate it.

05-31-2002, 02:40 PM	#7
beemk CLICK HERE Industry Role: Join Date: Jan 2002 Posts: 20,829	i just tested it and it worked except the error 404. i tried this the other day with just the error404 line and it still didnt work. any ideas?

05-31-2002, 07:09 PM	#9
beemk CLICK HERE Industry Role: Join Date: Jan 2002 Posts: 20,829	thanks once again 5eyes