There are many different reasons the tubes do well in search.
Factors include visitor retention, massive search footprint, affiliate programs that encourage direct linking to pages on the sites without an affiliate id - eg: Hubtraffic
However one factor that can be tackled by programs and affiliates alike is the use of search engine friendly urls. Another is the correct use of meta tags and schema.
Search Engine Friendly URLs
I am working on a search based project at the moment and am surprised by the number of big name sites that do not use search engine friendly urls.
This is giving the tubes a free kick, a leg up which is so easy to fix.
There is one site that I promote which is one of the biggest pay sites in the industry, yet for all of their archives and videos they use urls like the one below:
www.domain.com/video.php?slug=fucked_by_classmate_1461680
rather than
www.domain.com/video/fucked_by_classmate_1461680
Not only is the search engine friendly URL better for users to understand, it's also better for search engines to understand.
Markup, Meta Tags and Schema
When you're presenting data to search engines you want the search engine to be able to understand what your page is about. While modern search engines are good at doing this without reading meta tags, the use of meta tags and schema can greatly help rankings.
I've come across hundreds of smaller pay sites that use absolutely no schema at all, especially on pages where schema could help search engines understand the relationship of content to the site.
I've also come across several very large pay sites that have hundreds and sometimes thousands of pages with exactly the same meta tag descriptions and titles as their home page.
The use of search engine friendly URLs and correct use of meta tags and schema can greatly enhance search rankings. It might not completely beat the massive tubes like Pornhub but it can help boost rankings and ensure that Google and other search engines understand pages better thus being able to serve relevant results to search queries.
For more information on schema and how you can use it read the link below.
A Guide To Schema Markup & Structured Data SEO Opportunities By Site Type