Hard to answer without knowing what kind of site you will have.
For big programs with multiple sites NATS is the best platform, which can handle many billers.
I'll assume you'll run a small site yourself, rather than creating a bigger company with employees, accounting, etc.
Many affiliates will avoid promoting a custom built program that handles payments themselves. Burnt too many times with inaccurate or tricky stats, sales not being tracked properly, and not getting paid. Yes I'm sure that won't happen with Your Program. But such promises mean nothing, affiliates can't know in advance which programs will go south within a few years. I think 99% of the small programs mean well, but that's not enough.
Running your program in NATS and paying affiliates yourself is a bit better, because at least the stats tracking will work correctly.
For small sites owned by a single person, I'd only promote them if their affiliate program is operated through CCBILL, to avoid all risks like the above. My hits will be surely tracked and I will be surely paid. Even if the owner starts doing drugs, needs to pay their family member's hospital bill, gets in an accident, or disappears in the jungle during a treasure hunt.
And I'll also be paid if I only make $30 in two years. Mind you that some affiliate doing $30 with your program might be doing $3000 with another, they're just not pushing your program in particular. It's a bad practice to dismiss such affiliates as "not worth the time/energy", be unresponsive or rude to them. Many programs make that mistake, and the result they get is that some big affiliates avoid them completely, or send way less traffic than they could. A program treating small affiliates shitty is a warning sign to big affiliates.
A middle ground could be using NATS with a CCBILL paid feature. Then NATS handles tracking, and CCBILL handles affiliate payouts.
https://gfy.com/fucking-around-and-b...dam-promo.html
International affiliate payments can be a headache. Different affiliates around the world need different payment methods, payment service providers go down, your banks might kick you out for sending "suspicious" payments all over the world, payments don't arrive sometimes. It's a lot of help if you don't have to worry about these things, and CCBILL takes care of all that.
The hardest part about running an affiliate program is to get affiliates and convince them to send their traffic to you. Now if you're sure that it won't be a problem for you and you have affiliates lined up already, you're free to build your program in any way you want.
One of the tasks of the affiliate manager would be actually to GET you affiliates. They'll also have to talk to them, find out what they want and what their problems are. If he can deliver them these things, they'll send you more traffic and sales. The affiliate manager has to cooperate with your developer and you to set up partnerships/tours/promotools/etc. Solve the problems that come up, to get more traffic and better conversions.
Most affiliate managers are not developers, you'll need someone who knows business / marketing, is good with people, and has a little tech knowledge. The best is if they're already in the industry and have a good reputation.
If you're building a phone sex affiliate program it's a different story, many of the things above might not apply. The tools/platforms for paysite affiliate programs might not work. Most of the affiliates promoting phone sex won't be the same who promote paysites/cam sites. I don't think there are too many phone sex affiliate programs to choose from (just guessing). So you might actually have it easier and have a great chance of attracting big affiliates who have phone sex traffic.