Tech PHP Traffic Trading Script - Custom Logic Needed

[phil-flash]

I am writing a traffic trading script from scratch, and after diving into this rabbit hole, realized my logic was flawed and needed to be reconsidered.

I thought I could write one big be all end all script for a network of internal free sites, but realized that tracking and querying becomes complex, or maybe even impossible.

I am thinking something like this can only track one domain at a time, each free site needs it's own database and copy of the script. Is this correct?

Anyone smarter than me know or have ideas?

[sarettah]

Originally Posted by phil-flash

I am writing a traffic trading script from scratch, and after diving into this rabbit hole, realized my logic was flawed and needed to be reconsidered.

I thought I could write one big be all end all script for a network of internal free sites, but realized that tracking and querying becomes complex, or maybe even impossible.

I am thinking something like this can only track one domain at a time, each free site needs it's own database and copy of the script. Is this correct?

Anyone smarter than me know or have ideas?

There really is no reason you couldn't do it from one script and database.

If your sites are all on the same server then easy. If distributed across servers then it is more complex but can still be done.

.

[k0nr4d]

Originally Posted by phil-flash

I am thinking something like this can only track one domain at a time, each free site needs it's own database and copy of the script. Is this correct?

No, you can run the traffic all to one mysql table as long as it's indexed correctly. If you want to 'shard' it, you can do a separate table per site but when you start doing that any changes to the traffic table as you build out your script you've gotta do X times instead of once.

I would start it as one table and if it becomes a bottleneck look into ways of splitting it up.

[phil-flash]

Thanks for the replies. I powered through and figured it out.

One db and three tables.

IP TABLE: so I don't insert the same ip multiple times and add size to the db

CLICKS TABLE: records validated trade traffic clicks on inbound. Recording the from site, to site, unique ip id to track for raw/unique,

TRADES TABLE: free site trade data, I.e. domain name, url, id, and account contact info.

I never built anything like this before. It's pretty intense, frustrating, yet satisfying once you break through. The queries were the biggest challenge. "Get all clicks to this domain from all trade domains, except this domain, count clicks, sort count array, create link list"

It's pretty cool though, a one page wonder that I can include a top any free site on my server. It prints out a list that can later be echoed anywhere on the site.

Sorry for the brag, it's 3:30 am, I am jacked on coffee, and ecstatic with php success-ism!

Maybe the above logic helps someone...

[k0nr4d]

Make sure to check your queries using EXPLAIN

So l ike
EXPLAIN SELECT * FROM clicks WHERE foo = 'foo' AND bar = 'bar'

This will help you optimize your indexes and find bottlenecks. Avoid using "OR" in your queries as that generates a temporary table IIRC and will buttfuck you when the table gets too large.

[plsureking]

i watch a few of these kinds of videos each week to stay fresh and learn new tricks. one tweak can save resources and speed things up. youtube is full of tutorial videos lol



#

[brassmonkey]

i would like to see a link trade script. do you do custom work? pm me if you do. also you have my email

[TACNet]

For scaleablitiy, I'd be careful about writing your data directly to your MySql tables

We had the same issue when putting together our affiliate tracking scripts. It only takes one bot to start hammering your pages and it will down your DB server

We found a far better approach is to write everything to a log file first and then setup a cron script to process that log file every 15 minutes or whatever. That way you can filter out duplicates, bot scripts etc etc

Hope that helps

[Klen]

Originally Posted by TACNet

For scaleablitiy, I'd be careful about writing your data directly to your MySql tables

We had the same issue when putting together our affiliate tracking scripts. It only takes one bot to start hammering your pages and it will down your DB server

We found a far better approach is to write everything to a log file first and then setup a cron script to process that log file every 15 minutes or whatever. That way you can filter out duplicates, bot scripts etc etc

Hope that helps

I sent million hits in single day to my tracking script and there was no issue at all. Tho, i have delayed processing so that help. But your approach is good as well, it's like crude cache system.

[k0nr4d]

Originally Posted by TACNet

For scaleablitiy, I'd be careful about writing your data directly to your MySql tables

We had the same issue when putting together our affiliate tracking scripts. It only takes one bot to start hammering your pages and it will down your DB server

We found a far better approach is to write everything to a log file first and then setup a cron script to process that log file every 15 minutes or whatever. That way you can filter out duplicates, bot scripts etc etc

Hope that helps

With that reasoning, it would be better to use a mysql memory table as a buffer and insert into that, as if you are inserting into a log file you will have several issues.
- Constant IO Writes
- Write-locks on the log, since only 1 process can write to it at once.

[Klen]

Originally Posted by k0nr4d

With that reasoning, it would be better to use a mysql memory table as a buffer and insert into that, as if you are inserting into a log file you will have several issues.
- Constant IO Writes
- Write-locks on the log, since only 1 process can write to it at once.

Yep, i used similar approach with other type of script, till i switched to memcache which is far smoother method to deal with it.

[k0nr4d]

Originally Posted by Klen

Yep, i used similar approach with other type of script, till i switched to memcache which is far smoother method to deal with it.

Memcached has its uses of course but for something like this memory table is PERFECT because you can do something like

INSERT INTO `table` SELECT * FROM `buffer_table`; TRUNCATE `buffer_table`;

Making it very easy to migrate the data

[dave90210]

Originally Posted by k0nr4d

Constant IO Writes

https://en.wikipedia.org/wiki/Object...ed_programming
https://en.wikipedia.org/wiki/Object...tional_mapping
https://en.wikipedia.org/wiki/Message_broker

[plsureking]

Originally Posted by TACNet

For scaleablitiy, I'd be careful about writing your data directly to your MySql tables

We had the same issue when putting together our affiliate tracking scripts. It only takes one bot to start hammering your pages and it will down your DB server

We found a far better approach is to write everything to a log file first and then setup a cron script to process that log file every 15 minutes or whatever. That way you can filter out duplicates, bot scripts etc etc

Hope that helps

why aren't you replicating to mirror servers? front-end users shouldn't be hitting your primary db server.

i use flat file data for almost everything. i only use mysql if a table gets huge.

traffic stats definitely don't deserve mysql lol

#

[Klen]

Originally Posted by plsureking

why aren't you replicating to mirror servers? front-end users shouldn't be hitting your primary db server.

i use flat file data for almost everything. i only use mysql if a table gets huge.

traffic stats definitely don't deserve mysql lol

#

Yep, that is correct too - most of trade scripts were based on file storage, only EPT and FTT2 was based on mysql.

[CrazyMartin]

traffic trading scrips

ancient forgotten technology.....

you guys are funny

mysql for traffic trading scripts

hire archaeologists and start digging around 2000
then find out how huge sites manage their traffic traders on amd k6 servers 64mb - 512mb of memory

https://web.archive.org/web/20010205....com/info.html blindio traffic trading script $2k for copy
most famous ucj? https://web.archive.org/web/20000819...rinfo.com/ucj/ $1.2k

[sandman!]

Tgp Software.com - Professional Traffic Managment Solutions wonder if they still sell the script

[Zug]

Whatever happened to Trade Expert? Trade Expert .. seems they upgraded recently, about a year and a half ago. But its seems their stuff is broken; can't register.. Nice script when I used it before. Just was wondering.

Trade script is still needed imo, way to phil!

[sandman!]

I don’t think there are enough paying customers for anyone to do a trade script anymore.

There used to be a ton of options

[AmeliaG]

Originally Posted by plsureking

i watch a few of these kinds of videos each week to stay fresh and learn new tricks. one tweak can save resources and speed things up. youtube is full of tutorial videos lol



#

That is cool.

YouTube Premium is the only streaming service I always have. I loooooove educational YouTube.

[plsureking]

Originally Posted by AmeliaG

That is cool.

YouTube Premium is the only streaming service I always have. I loooooove educational YouTube.

Linkedin learning is pretty good. i caught up on what the corporate idiots are doing during Rona. LI bought the Lynda library and produce their own tutorial courses.

ps i pay for YT Premium too. the whole family uses it all day all night.

#