I found a METHOD to detect sites sending BOT NET TRAFFIC...

[ipolic]

<rant_mode>I am thinking for last 2 hours to write here or not, because probably all what I will read in posts below will be newb comments and post whore messages, but I still decided to give it a go.</rant_mode>

Basically I was working on project (not relevant to adult) and I was struck by this brilliant idea.

What is the biggest traffic related issue in industry? How to filter out (detect) sites who send you hitbot(bot nets) traffic. I don't speak about old bots which average script and couple jquery tricks can stop, but about advanced, malware infected browsers, visiting your website in background, not clicking on ads (some even do that) but at the end -> not buying anything.

My idea is simple, I tested it and it works. It analyses traffic stream from one source and compares it to others, when you see difference in value, YOU KNOW that site is sending you bot traffic. Bigger the difference, more bots they send. Basically it compares some values from 1 site to values of averages from other sites, and no it is not cookies, javascript, country, proxy and other things any half decent nowaday bot can "hijack".

Does anyone have any smart idea what to do with discovery? How to monetize it? Offer it as service or sell it to someone? I know it would be best to give it away for free + give you free 500 bucks on paxum for whores, but guess what, that's not gonna happen. Anyone interested to make me an offer I cannot refuse for my discovery?

Broker networks, huge tube sites or anyone else who deals with large pool of traffic can have HUGE benefit of this!

[olivierx]

Originally Posted by ipolic

<rant_mode>I am thinking for last 2 hours to write here or not, because probably all what I will read in posts below will be newb comments and post whore messages, but I still decided to give it a go.</rant_mode>

Basically I was working on project (not relevant to adult) and I was struck by this brilliant idea.

What is the biggest traffic related issue in industry? How to filter out (detect) sites who send you hitbot(bot nets) traffic. I don't speak about old bots which average script and couple jquery tricks can stop, but about advanced, malware infected browsers, visiting your website in background, not clicking on ads (some even do that) but at the end -> not buying anything.

My idea is simple, I tested it and it works. It analyses traffic stream from one source and compares it to others, when you see difference in value, YOU KNOW that site is sending you bot traffic. Bigger the difference, more bots they send. Basically it compares some values from 1 site to values of averages from other sites, and no it is not cookies, javascript, country, proxy and other things any half decent nowaday bot can "hijack".

Does anyone have any smart idea what to do with discovery? How to monetize it? Offer it as service or sell it to someone? I know it would be best to give it away for free + give you free 500 bucks on paxum for whores, but guess what, that's not gonna happen. Anyone interested to make me an offer I cannot refuse for my discovery?

Broker networks, huge tube sites or anyone else who deals with large pool of traffic can have HUGE benefit of this!

I managed to find a better, more simple method to detect bot traffic. Nowdays most malware infected traffic are clicking on site, clicking on ads therefor acting 'normal'

The best, most simple way i found to find out which site are sending bot traffic.

You put a transparant image on top of your site before all links, in the corner, so that no one normally click it, you make it clickable. should go to any page where the only way to go is this transparent image. You setup so that it record every ip of who visit this page, record the referrer, you search the ip/referer on your stats to find out where the hit come from.

Usiung this system i am able to block most of the bad source from traffic exchange. Of course its not good proof enough to ban someone from a traffic network, but, its enough proof for me to filter his traffic out of my sites.

[Serge Litehead]

Originally Posted by olivierx

I managed to find a better, more simple method to detect bot traffic. Nowdays most malware infected traffic are clicking on site, clicking on ads therefor acting 'normal'

The best, most simple way i found to find out which site are sending bot traffic.

You put a transparant image on top of your site before all links, in the corner, so that no one normally click it, you make it clickable. should go to any page where the only way to go is this transparent image. You setup so that it record every ip of who visit this page, record the referrer, you search the ip/referer on your stats to find out where the hit come from.

Usiung this system i am able to block most of the bad source from traffic exchange. Of course its not good proof enough to ban someone from a traffic network, but, its enough proof for me to filter his traffic out of my sites.

great idea. but since you shared it you may need to adjust the strategy later on, especially when more sites integrate it. bot runners may be able to come around this somehow

[ipolic]

Ok idea, but bots (from my knowledge) have workarounds for this for ages. Don't wanna underrate your method, just saying there is much better way. Your way is also limited for site owners, my way is not limited, you can analyze entire traffic pool with it (broker systems etc).

Originally Posted by olivierx

I managed to find a better, more simple method to detect bot traffic. Nowdays most malware infected traffic are clicking on site, clicking on ads therefor acting 'normal'

The best, most simple way i found to find out which site are sending bot traffic.

You put a transparant image on top of your site before all links, in the corner, so that no one normally click it, you make it clickable. should go to any page where the only way to go is this transparent image. You setup so that it record every ip of who visit this page, record the referrer, you search the ip/referer on your stats to find out where the hit come from.

Usiung this system i am able to block most of the bad source from traffic exchange. Of course its not good proof enough to ban someone from a traffic network, but, its enough proof for me to filter his traffic out of my sites.

[ipolic]

Originally Posted by holograph

great idea. but since you shared it you may need to adjust the strategy later on, especially when more sites integrate it. bot runners may be able to come around this somehow

There are always workarounds, but in this case it would be more expensive and non reliable to them -> it's not worth it -> it would make more expense then profit.

[TeenCat]

so you compare other results with some site, of course your site, but what if your site sucks and someone may have much better stats? for example i see it like you have somewhere ratio 1:400, i can do 1:10, so you decide that it must be bot or something because it is so far from your results?

[plaster]

Originally Posted by TeenCat

so you compare other results with some site, of course your site, but what if your site sucks and someone may have much better stats? for example i see it like you have somewhere ratio 1:400, i can do 1:10, so you decide that it must be bot or something because it is so far from your results?

He has virtual girl with single malt scotch in hand with question...

Would you like free single malt hun?

If results of click are 50/50 it is bot traffic.

[NickBaer]

My top three aged sites have 65% BOT traffic.

My mid-aged sites are 35% BOT traffic.

My newest site has 0% BOT traffic.

Are you buying Traffic? Are you monitoring BOT Traffic?

I put a couple of $_SERVER[] commands in my footer. For every hit, I get:

20160126210314, /MJOVideos.php, 66.249.65.41, , ^ Mozilla/5.0 (compatible; Googlebot/2.1; +Googlebot - Search Console Help) |66.249.65.41,US,United States,CA,California,Mountain View,94043,America/Los_Angeles,37.42,-122.06,807

^^ With a simple piece of data, I can measure up to 65% BOT traffic to my sites, with this simple {HP code:

date("YmdHis").', '. $_SERVER['PHP_SELF'].', '.$_SERVER["REMOTE_ADDR"].', '.$_SERVER['HTTP_REFERER'].', '.$_SERVER['QUERY_STRING'].'^ '.$_SERVER['HTTP_USER_AGENT'] (plus geoip lookup) . PHP_EOL;

And then I Save ('a' append) to a txt file. I change it monthly.

[rowan]

Originally Posted by NickBaer

My top three aged sites have 65% BOT traffic.

[...]

20160126210314, /MJOVideos.php, 66.249.65.41, , ^ Mozilla/5.0 (compatible; Googlebot/2.1; +Googlebot - Search Console Help) |66.249.65.41,US,United States,CA,California,Mountain View,94043,America/Los_Angeles,37.42,-122.06,807

^^ With a simple piece of data, I can measure up to 65% BOT traffic to my sites, with this simple {HP code:

Your example shows Googlebot, which is a web crawler - it doesn't claim to be anything other than a bot. The OP is talking about zombie machines - consumer computers that are (unknowingly) used as proxies to simulate real humans, usually for click fraud.

How are you measuring your 65%? Anything with "bot" in the user-agent?

[dillfly2000]

How do sites like JADS afford to keep paying these proxy botters? I would desperately want to crack down and ban any site accused of artificial ad clicks.

[olivierx]

Originally Posted by holograph

great idea. but since you shared it you may need to adjust the strategy later on, especially when more sites integrate it. bot runners may be able to come around this somehow

I bet they will, i have other catcher for the botter on my sites whichc i dont post here, i had 70% bot traffic from traffic buying sites like trafficholder,exoclick,ero-advertising with my filter, with blocking bad source, i think i now only have under 10% of my traffic that is zombies.

The problem can't be solved in one day, i did alot of stats analyze to be able to block the bad sources, it make me sad that some of the big company like Exoclick, Ero-Advertising have lot of zombie traffic... Some of the site that are sending over 60% botnet traffic are even ''featured'' some times... I'm not even pro webmaster or pro technician guy and i was able to catch on my first day of stats analyzing from exoclick traffic over 5 different sources whicih send ALOT of bot traffic.. I do honestly think that some people are making thousand's of dollars every day from bot traffic...

Any big ads company that want to catch bot traffic can do it very easily. ( for site which send over 50% ) but, the real truth is that i highly DOUBT they want to catch them. As long as they make traffic look like real, they don't care and are happy to let them send their traffic so they can continue to advertise thing like ''OVER 5 BILLION AD IMPRESSION''

[shake]

There are some great methods being used by cloudflare to stop ddos attacks.

[xXXtesy10]

welcome to 2001 have luck

[ravo]

Why not use this new method as a competitive advantage on your own ad network? Why would you want to share something like this?

[ipolic]

It would be exactly someone who would have HUGE benefit from my project.

Originally Posted by dillfly2000

How do sites like JADS afford to keep paying these proxy botters? I would desperately want to crack down and ban any site accused of artificial ad clicks.

[ipolic]

Originally Posted by ravo

Why not use this new method as a competitive advantage on your own ad network? Why would you want to share something like this?

We are too small players to monetize it good enough as someone who has much bigger traffic flow.

[Colmike9]

Edit: nvm

[NickBaer]

Originally Posted by rowan

Your example shows Googlebot, which is a web crawler - it doesn't claim to be anything other than a bot. The OP is talking about zombie machines - consumer computers that are (unknowingly) used as proxies to simulate real humans, usually for click fraud.

How are you measuring your 65%? Anything with "bot" in the user-agent?

Of course! Not-human hits. Useless.

Google crawls me all day long. Doesn't buy anything. Doesn't (always) put me on page 1.

There's BOTs from America, Europe, Asia.

All of that is click fraud.

[Colmike9]

Don't pay for traffic and ignore the bots, then..

[lezinterracial]

Thought you may like this.

[ITraffic]

Originally Posted by NickBaer

Of course! Not-human hits. Useless.

Google crawls me all day long. Doesn't buy anything. Doesn't (always) put me on page 1.

There's BOTs from America, Europe, Asia.

All of that is click fraud.

User-agent: *
Disallow: /

[PornDiscounts-V]

Don't buy traffic. Don't trade traffic. It really is that simple.

[8pt-buck]

Originally Posted by vvvvv

Don't buy traffic. Don't trade traffic. It really is that simple.

Best advice in this thread.

[CPA-Rush]

Originally Posted by olivierx

the real truth is that i highly DOUBT they want to catch them. As long as they make traffic look like real, they don't care and are happy to let them send their traffic so they can continue to advertise thing like ''OVER 5 BILLION AD IMPRESSION''

[slapass]

Originally Posted by olivierx

I managed to find a better, more simple method to detect bot traffic. Nowdays most malware infected traffic are clicking on site, clicking on ads therefor acting 'normal'

The best, most simple way i found to find out which site are sending bot traffic.

You put a transparant image on top of your site before all links, in the corner, so that no one normally click it, you make it clickable. should go to any page where the only way to go is this transparent image. You setup so that it record every ip of who visit this page, record the referrer, you search the ip/referer on your stats to find out where the hit come from.

Usiung this system i am able to block most of the bad source from traffic exchange. Of course its not good proof enough to ban someone from a traffic network, but, its enough proof for me to filter his traffic out of my sites.

I had something like this back in the day. It works pretty well. I can see where a larger image more in the text area might have a better reading. I used a really small image and quit when i got worried about a Google penalty.

[CPA-Rush]

Originally Posted by NickBaer

My top three aged sites have 65% BOT traffic.

My mid-aged sites are 35% BOT traffic.

My newest site has 0% BOT traffic.

Are you buying Traffic? Are you monitoring BOT Traffic?

I put a couple of $_SERVER[] commands in my footer. For every hit, I get:

20160126210314, /MJOVideos.php, 66.249.65.41, , ^ Mozilla/5.0 (compatible; Googlebot/2.1; +Googlebot - Search Console Help) |66.249.65.41,US,United States,CA,California,Mountain View,94043,America/Los_Angeles,37.42,-122.06,807

^^ With a simple piece of data, I can measure up to 65% BOT traffic to my sites, with this simple {HP code:

date("YmdHis").', '. $_SERVER['PHP_SELF'].', '.$_SERVER["REMOTE_ADDR"].', '.$_SERVER['HTTP_REFERER'].', '.$_SERVER['QUERY_STRING'].'^ '.$_SERVER['HTTP_USER_AGENT'] (plus geoip lookup) . PHP_EOL;

And then I Save ('a' append) to a txt file. I change it monthly.

anyone can use google u-agent, but this google ip this not a bot traffic.

[slapass]

Originally Posted by ravo

Why not use this new method as a competitive advantage on your own ad network? Why would you want to share something like this?

Because an ad network couldn't make money without the bots.... oh never mind.

J/K. I have used Ravo's traffic and have no memory of how it went so it must have gone fine.

[adultmobile]

Originally Posted by vvvvv

Don't buy traffic. Don't trade traffic. It really is that simple.

Don't buy traffic from who write you "Sir " and "God Bless" and "more Power" in the emails.

[Barry-xlovecam]

There are SaaS products out there ...

Device Based Authentication and Online Risk Solutions | iovation
Block Bots and Stop Web Scraping | Distil Networks

These people are not into securing link-trading because that is a rather narrow market.

Google has extensive click-bot protection now on AdWords and Adsense.

The problem with your desired uptake of ad network interest; is that they pay and charge by the gross delivery and not by the clicks that the advertiser receives. This may only be limited to the bulk traffic that they sell -- 40% of it is total crap but the price that it is sold for has to be factored in with the cost and returns of CPC traffic.

Why not try selling the idea as a SaaS service at a monthly cost? Compile an api that uses the factors that you have discovered ''your secret sauce'' and have the api allow or deny the clicks and report the findings back to the subscriber. Find out just how many webmasters will pay by the month for this type SaaS.

[ravo]

Originally Posted by slapass

Because an ad network couldn't make money without the bots.... oh never mind.
.

Yeah, there's an object lesson here. Sure, some ad networks may not make a good effort to stop bots, but for any ad network with a long term strategy, it would be suicide to not make every possible effort to stop bots.

Ultimately, every ad network depends on repeat buyers to stay in business. The pool of advertisers willing to pay for traffic keeps getting smaller every year, and if you piss one of them off with shit traffic, they will NEVER be back. They have long memories.

But, if you provide steady, productive, profitable traffic they re-buy for YEARS. I know. I have clients that have been buying continuously for 10+ years.

So, really, your statement should have been: "Because an ad network couldn't make money WITH the bots" ;)

[ipolic]

This is actually thing that crossed my mind, but I cannot afford "tons" of cash for advertising which is needed nowadays, I am already in other investments. This would be really nice SaaS with monthly subscription for any average webmaster out there, including but NOT LIMITED to adult.

Originally Posted by Barry-xlovecam

There are SaaS products out there ...

Device Based Authentication and Online Risk Solutions | iovation
Block Bots and Stop Web Scraping | Distil Networks

These people are not into securing link-trading because that is a rather narrow market.

Google has extensive click-bot protection now on AdWords and Adsense.

The problem with your desired uptake of ad network interest; is that they pay and charge by the gross delivery and not by the clicks that the advertiser receives. This may only be limited to the bulk traffic that they sell -- 40% of it is total crap but the price that it is sold for has to be factored in with the cost and returns of CPC traffic.

Why not try selling the idea as a SaaS service at a monthly cost? Compile an api that uses the factors that you have discovered ''your secret sauce'' and have the api allow or deny the clicks and report the findings back to the subscriber. Find out just how many webmasters will pay by the month for this type SaaS.