GoFuckYourself.com - Adult Webmaster Forum

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)
-   Fucking Around & Business Discussion (https://gfy.com/forumdisplay.php?f=26)
-   -   Tech PHP Traffic Trading Script - Custom Logic Needed (https://gfy.com/showthread.php?t=1353217)

phil-flash 03-16-2022 03:37 PM

PHP Traffic Trading Script - Custom Logic Needed
 
I am writing a traffic trading script from scratch, and after diving into this rabbit hole, realized my logic was flawed and needed to be reconsidered.

I thought I could write one big be all end all script for a network of internal free sites, but realized that tracking and querying becomes complex, or maybe even impossible.

I am thinking something like this can only track one domain at a time, each free site needs it's own database and copy of the script. Is this correct?

Anyone smarter than me know or have ideas?

sarettah 03-16-2022 05:06 PM

Quote:

Originally Posted by phil-flash (Post 22979767)
I am writing a traffic trading script from scratch, and after diving into this rabbit hole, realized my logic was flawed and needed to be reconsidered.

I thought I could write one big be all end all script for a network of internal free sites, but realized that tracking and querying becomes complex, or maybe even impossible.

I am thinking something like this can only track one domain at a time, each free site needs it's own database and copy of the script. Is this correct?

Anyone smarter than me know or have ideas?

There really is no reason you couldn't do it from one script and database.

If your sites are all on the same server then easy. If distributed across servers then it is more complex but can still be done.

.

k0nr4d 03-17-2022 12:33 AM

Quote:

Originally Posted by phil-flash (Post 22979767)
I am thinking something like this can only track one domain at a time, each free site needs it's own database and copy of the script. Is this correct?

No, you can run the traffic all to one mysql table as long as it's indexed correctly. If you want to 'shard' it, you can do a separate table per site but when you start doing that any changes to the traffic table as you build out your script you've gotta do X times instead of once.

I would start it as one table and if it becomes a bottleneck look into ways of splitting it up.

phil-flash 03-17-2022 01:31 AM

Thanks for the replies. I powered through and figured it out.

One db and three tables.

IP TABLE: so I don't insert the same ip multiple times and add size to the db

CLICKS TABLE: records validated trade traffic clicks on inbound. Recording the from site, to site, unique ip id to track for raw/unique,

TRADES TABLE: free site trade data, I.e. domain name, url, id, and account contact info.

I never built anything like this before. It's pretty intense, frustrating, yet satisfying once you break through. The queries were the biggest challenge. "Get all clicks to this domain from all trade domains, except this domain, count clicks, sort count array, create link list"

It's pretty cool though, a one page wonder that I can include a top any free site on my server. It prints out a list that can later be echoed anywhere on the site.

Sorry for the brag, it's 3:30 am, I am jacked on coffee, and ecstatic with php success-ism!

Maybe the above logic helps someone...

k0nr4d 03-17-2022 01:37 AM

Make sure to check your queries using EXPLAIN

So l ike
EXPLAIN SELECT * FROM clicks WHERE foo = 'foo' AND bar = 'bar'

This will help you optimize your indexes and find bottlenecks. Avoid using "OR" in your queries as that generates a temporary table IIRC and will buttfuck you when the table gets too large.

plsureking 03-17-2022 11:45 AM

i watch a few of these kinds of videos each week to stay fresh and learn new tricks. one tweak can save resources and speed things up. youtube is full of tutorial videos lol



#

brassmonkey 03-17-2022 11:50 AM

i would like to see a link trade script. do you do custom work? pm me if you do. also you have my email

TACNet 03-18-2022 07:26 AM

For scaleablitiy, I'd be careful about writing your data directly to your MySql tables

We had the same issue when putting together our affiliate tracking scripts. It only takes one bot to start hammering your pages and it will down your DB server

We found a far better approach is to write everything to a log file first and then setup a cron script to process that log file every 15 minutes or whatever. That way you can filter out duplicates, bot scripts etc etc

Hope that helps

Klen 03-18-2022 07:48 AM

Quote:

Originally Posted by TACNet (Post 22980517)
For scaleablitiy, I'd be careful about writing your data directly to your MySql tables

We had the same issue when putting together our affiliate tracking scripts. It only takes one bot to start hammering your pages and it will down your DB server

We found a far better approach is to write everything to a log file first and then setup a cron script to process that log file every 15 minutes or whatever. That way you can filter out duplicates, bot scripts etc etc

Hope that helps

I sent million hits in single day to my tracking script and there was no issue at all. Tho, i have delayed processing so that help. But your approach is good as well, it's like crude cache system.

k0nr4d 03-18-2022 07:56 AM

Quote:

Originally Posted by TACNet (Post 22980517)
For scaleablitiy, I'd be careful about writing your data directly to your MySql tables

We had the same issue when putting together our affiliate tracking scripts. It only takes one bot to start hammering your pages and it will down your DB server

We found a far better approach is to write everything to a log file first and then setup a cron script to process that log file every 15 minutes or whatever. That way you can filter out duplicates, bot scripts etc etc

Hope that helps

With that reasoning, it would be better to use a mysql memory table as a buffer and insert into that, as if you are inserting into a log file you will have several issues.
- Constant IO Writes
- Write-locks on the log, since only 1 process can write to it at once.

Klen 03-18-2022 08:06 AM

Quote:

Originally Posted by k0nr4d (Post 22980531)
With that reasoning, it would be better to use a mysql memory table as a buffer and insert into that, as if you are inserting into a log file you will have several issues.
- Constant IO Writes
- Write-locks on the log, since only 1 process can write to it at once.

Yep, i used similar approach with other type of script, till i switched to memcache which is far smoother method to deal with it.

k0nr4d 03-18-2022 09:00 AM

Quote:

Originally Posted by Klen (Post 22980534)
Yep, i used similar approach with other type of script, till i switched to memcache which is far smoother method to deal with it.

Memcached has its uses of course but for something like this memory table is PERFECT because you can do something like

INSERT INTO `table` SELECT * FROM `buffer_table`; TRUNCATE `buffer_table`;

Making it very easy to migrate the data

dave90210 03-18-2022 11:33 AM

Quote:

Originally Posted by k0nr4d (Post 22980531)
Constant IO Writes

https://en.wikipedia.org/wiki/Object...ed_programming
https://en.wikipedia.org/wiki/Object...tional_mapping
https://en.wikipedia.org/wiki/Message_broker

plsureking 03-18-2022 11:45 AM

Quote:

Originally Posted by TACNet (Post 22980517)
For scaleablitiy, I'd be careful about writing your data directly to your MySql tables

We had the same issue when putting together our affiliate tracking scripts. It only takes one bot to start hammering your pages and it will down your DB server

We found a far better approach is to write everything to a log file first and then setup a cron script to process that log file every 15 minutes or whatever. That way you can filter out duplicates, bot scripts etc etc

Hope that helps

why aren't you replicating to mirror servers? front-end users shouldn't be hitting your primary db server.

i use flat file data for almost everything. i only use mysql if a table gets huge.

traffic stats definitely don't deserve mysql lol

#

Klen 03-18-2022 11:49 AM

Quote:

Originally Posted by plsureking (Post 22980636)
why aren't you replicating to mirror servers? front-end users shouldn't be hitting your primary db server.

i use flat file data for almost everything. i only use mysql if a table gets huge.

traffic stats definitely don't deserve mysql lol

#

Yep, that is correct too - most of trade scripts were based on file storage, only EPT and FTT2 was based on mysql.

CrazyMartin 03-18-2022 02:17 PM

traffic trading scrips

ancient forgotten technology.....

you guys are funny :)

mysql for traffic trading scripts :)

hire archaeologists and start digging around 2000 :)
then find out how huge sites manage their traffic traders on amd k6 servers :) 64mb - 512mb of memory :)

https://web.archive.org/web/20010205....com/info.html blindio traffic trading script $2k for copy :)
most famous ucj? https://web.archive.org/web/20000819...rinfo.com/ucj/ $1.2k

sandman! 03-18-2022 06:10 PM

Tgp Software.com - Professional Traffic Managment Solutions wonder if they still sell the script

Zug 03-19-2022 01:24 PM

Whatever happened to Trade Expert? Trade Expert .. seems they upgraded recently, about a year and a half ago. But its seems their stuff is broken; can't register.. Nice script when I used it before. Just was wondering.

Trade script is still needed imo, way to phil!

sandman! 03-19-2022 07:48 PM

I don’t think there are enough paying customers for anyone to do a trade script anymore.

There used to be a ton of options

AmeliaG 03-20-2022 01:14 AM

Quote:

Originally Posted by plsureking (Post 22980191)
i watch a few of these kinds of videos each week to stay fresh and learn new tricks. one tweak can save resources and speed things up. youtube is full of tutorial videos lol



#

That is cool.

YouTube Premium is the only streaming service I always have. I loooooove educational YouTube.

plsureking 03-20-2022 10:24 AM

Quote:

Originally Posted by AmeliaG (Post 22981307)
That is cool.

YouTube Premium is the only streaming service I always have. I loooooove educational YouTube.

Linkedin learning is pretty good. i caught up on what the corporate idiots are doing during Rona. LI bought the Lynda library and produce their own tutorial courses.

ps i pay for YT Premium too. the whole family uses it all day all night.

#


All times are GMT -7. The time now is 07:16 AM.

Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc