GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Transcoding in 2018? What are the must haves?

VRR · 04-19-2018, 06:51 AM

There is no silver bullet for this. The amount of hardware needed would depend entirely on the time you are willing to spend transcoding. The thing is that while CPUs continue to advance quite fast with every next generation, transcoding continues to be extremely taxing and slow process. GPU transcoding, on the other hand, continues to be a bad alternative minaly due to the quality/bitrate limitations. The files transcoded with a GPU remain too big for online use. NVENC (Nvidia) does it better than QuickSync (Intel). However, Nvidia puts artificial limitation on their customer grade hardware so that only 2 simultaneous transcodes can be ran at any given moment. Their PRO cards dont have this limit but at prices of a few K $ per card, it is simply not a viable technology to use.

Modern devices all support HLS now. That is the adaptive bitrate streaming thing that auto adjusts the quality depending on the bandwidth available to the device, so that playback interruptons could be eliminated as much as possible. It's MPEG + AAC inside, which is playable on pretty much everything produced in the past few years. Alternatively, to reach as many devices as possible, you can go with MP4, keeping an eye on the so called levels to ensure maximum device compatibility. The output would at best be in 3 resolutions: 360p/480p with probably a main profile 3.1 level for older devices, and then 720p and 1080p which should do well at 4.1 main profile.

H265 is what should come next, after H264 is no longer the deal. However, at present if you try to encode your library to H265 it would take literally forever. Then, H265 is not supported by the hardware of most devices from the lower end yet. So they will not be able to decode it for smooth playback.

It's up to you to decide how you will deliver the content (Static MP4s, or HLS ABRs), but in both cases the process of transcoding the whole library will be very very lengthy. It will require multiple servers and some way for you to manage which one transcodes what, so you can spread the load on them.
Intel E3s are your friend here. Due to limitations in libx264, the more cores you have for transcoding, the less gain per core you get in terms of speed. There is an overhead on the whole mutli-threading process. On the other hand, x264 benefits from faster cores, and E3s are fast. So money/performance they are a great choice.

In any case, it would take long. We have a customer that came to us with 160GB of 720p MP4s they wanted to make 480p versions of. It took 26 hours on an E3-1245v2 or v3 (not sure). I believe we could have lowered it to ~20 hours if we were applying less compression, but then the whole idea of 480p nowadays is to use less bandwidth so...

04-19-2018, 06:51 AM
VRR Confirmed User Industry Role: Join Date: Jul 2017 Location: Worldwide Posts: 37	There is no silver bullet for this. The amount of hardware needed would depend entirely on the time you are willing to spend transcoding. The thing is that while CPUs continue to advance quite fast with every next generation, transcoding continues to be extremely taxing and slow process. GPU transcoding, on the other hand, continues to be a bad alternative minaly due to the quality/bitrate limitations. The files transcoded with a GPU remain too big for online use. NVENC (Nvidia) does it better than QuickSync (Intel). However, Nvidia puts artificial limitation on their customer grade hardware so that only 2 simultaneous transcodes can be ran at any given moment. Their PRO cards dont have this limit but at prices of a few K $ per card, it is simply not a viable technology to use. Modern devices all support HLS now. That is the adaptive bitrate streaming thing that auto adjusts the quality depending on the bandwidth available to the device, so that playback interruptons could be eliminated as much as possible. It's MPEG + AAC inside, which is playable on pretty much everything produced in the past few years. Alternatively, to reach as many devices as possible, you can go with MP4, keeping an eye on the so called levels to ensure maximum device compatibility. The output would at best be in 3 resolutions: 360p/480p with probably a main profile 3.1 level for older devices, and then 720p and 1080p which should do well at 4.1 main profile. H265 is what should come next, after H264 is no longer the deal. However, at present if you try to encode your library to H265 it would take literally forever. Then, H265 is not supported by the hardware of most devices from the lower end yet. So they will not be able to decode it for smooth playback. It's up to you to decide how you will deliver the content (Static MP4s, or HLS ABRs), but in both cases the process of transcoding the whole library will be very very lengthy. It will require multiple servers and some way for you to manage which one transcodes what, so you can spread the load on them. Intel E3s are your friend here. Due to limitations in libx264, the more cores you have for transcoding, the less gain per core you get in terms of speed. There is an overhead on the whole mutli-threading process. On the other hand, x264 benefits from faster cores, and E3s are fast. So money/performance they are a great choice. In any case, it would take long. We have a customer that came to us with 160GB of 720p MP4s they wanted to make 480p versions of. It took 26 hours on an E3-1245v2 or v3 (not sure). I believe we could have lowered it to ~20 hours if we were applying less compression, but then the whole idea of 480p nowadays is to use less bandwidth so... __________________ PUSHR CDN - High-performance content delivery network for fast growing projects Skype: Victor_NetStack