Would you check user uploaded videos against a copyrighted "do not use" list if you could?
 

So, if there was an easy and free way to automate checking your user-uploaded content against a master list of movies that are not authorised for distribution, would you implement it? This is an anonymous poll.

Note, by doing so, you wouldn't break DMCA rules since this is automated and not all-inclusive.

However, accepting a flagged video would be a different kettle of fish, meaning you knowingly opted to have that video shown knowing it was not allowed to be shown...

Youtube does similar things. Would this work for adult? Discuss...

What are the checks? Some type of embed into original file?

Yeah... I think it sounds good if it is thought through. That would also mean 1 software would have to embed the "flag".

The check would be:
1. user uploads their video
2. your system fingerprints the video
3. your system checks this fingerprint against a master remote database of content not authorised for distribution
4. if video is on the list, you reject it from showing on your tube.

I provide the tools by working with the developers of tube scripts to have this implemented to make this is transparent as possible, or provide a simple framework for you if you are a developer yourself.

who covers the liability for false positives ?

How would you compare an uploaded video to a database of copyrighted videos? If the illegal copy has a watermark or is edited/shortened/compressed then how do you know it's the same video as the original that is copyrighted? The filesize, source, and frames are all different.

What's stopping someone who wants to start an illegal tube from just removing this check? Or buying another tube script that doesn't have it?

How much would it slow down the upload process if you are running this check across millions of videos? This one is probably easy to get around.

Sounds like a great idea though for the people who want to run a legit tube without all the risk and hassle of illegal uploading.

It doesn't matter if the video uploaded is a clip of the original, a compressed version, a watermarked version or anything. Simply put, ff the video uploaded is a match, it is a match.

Nothing at all - this is an opt-in option, that's all.

It takes ~15 seconds to watermark a 2hour video, then about 10milliseconds to check that against over 2 million videos, so I'd say it won't slow down the process to any significant length.

That is the idea :winkwink:

What liability? If you are the owner of the tube and you allow video uploads, you add a clause to say the uploaded video will be automatically checked against a database of copyrighted content and you reserve the right to refuse uploads of content that matches.

Would you mind sharing how that is accomplished? As far as I know if a media is changed then there isn't really much of a programmatic way of comparing it to the original, especially if compression is involved where every pixel is changed. Besides having a human look at them..

I bet simply adding that clause to the upload form would get rid of most of the pirates lol. Throw in something about being "reported to authorities".

Yes I would. I would also like to thank you for taking the lead on this.:thumbsup

I've spent a *lot* of personal time over the last year doing this and aim to do it not-for-profit. Also, if I open up how it is done, then pirates will figure out a way around it, so sorry; no - the concept will remain closed.

However, I'll get a proof-of-concept set up with say the entire library of a major tube (eg ~150k videos) so you can test away till your heart's content to see if you can get around it by clipping, compressing etc. If a match is found, it will show you the URL where the video is at. Will that do?

:thumbsup

Not trying discredit what you're doing, just some questions I had. I think you should definitely setup that test and have people here try to get past it so you can maybe improve it.

Isn't this system already available?

I'd probably implement it into mechbunny as a toggleable feature if such a system existed, had an API that could be used, and actually functioned well enough that it didn't give too many false positives... you would really need the support of tube script authors to implement this into their scripts for it to be a success, but there has to be a way to disable such checking on the script end as alot of tube site owners are affiliate programs that are adding thier own full length content to their own tubes. You can't do this with simple pattern recognition either because a trailer or shorter clip of a full video would have many of the same frames. Would have to factor in video length etc.

Actually, as long as the library is being updated constantly this isn't an issue. Tubes are the sources of the videos that get uploaded to tubes anyways. If some big tube supports this and someone uploads some cropped/clipped/compressed/whatever video, then people will download this and upload it to other tubes.

I believe Manwin has this available as a per-video payable service. However, this only applies to videos uploaded to their tube network.

This is an attempt to allow content producers to fingerprint their content and add it to a master 'do-not-use' list that others are free to check against.

Since it costs nothing to me (except cpu cycles, bandwidth, and sugar for my brain), yet has the potential to help those who care to avoid having copyrighted works on their tubes, I don't see the need to charge for it like Manwin do.

Of course, it does nothing to stop those who couldn't care less what content is on their tubes... that is phase 2 (which is ongoing 24/7/365) and is much more complicated and costly to me, and thus will end up being a pay-for service to allow content producers to automatically police the tubes to have their content removed from offending sites.

Thank K0nr4d for the input - indeed for this to work, tube authors need to add it for sure. All I can provide are the tools (eg a compiled C script to fingerprint the movie and then developer notes on what to do with that). FWIW, false positives in my tests are extremely rare, but I'll get a POC up for people to test out before I make any more assumptions.

The clip concept is interesting - a content producer with a 60 minute movie adds that to the copyright db. They then release a 5 minute clip that does the rounds on the tubes. The 5-minute clip would of course match the 60-minute movie and get flagged, which isn't at all what we want.

hmmm...

The fingerprint of both movie and clip contains the video lengths, so it's possible to only raise a flag if the queried fingerprint is > xx% of the db fingerprint.
In that case, xx% would need to be configurable by the tube owner. Of course, the tube script could also allow flagging of certain user's to not be checked.. this is all down to the tube author to implement...

And herein lies the problem!
Manwin has pretty much assimilated all the major tubes and as they offer this as a paid-for service (really quite expensive paid for service to boot), they will never come on board with this....

Too late to edit my post, but:

perhaps better would be for the db server to reply something like:
result: (bool)
match_owner: {copyright owner contact or something}
match_video_length: (int)secs
query_video_length: (int)secs

That would give the tube script the freedom to do as they please?

All this is the reason why I said "discuss" - feedback, ideas are all good things :thumbsup

Yep something like that would work. You could go a little further then a boolean result though.
You could spit back:
result: (int)percentage_match

and then let the script end decide "if its more then x% match, flag as possible copyright", "if its more then y% match, delete immediately"

Also you'd need to compare more then just a few frames. Alot of videos have a black screen or copyright notice or intro video for the first x seconds of the clip, which could generate false positives as well since it could match quite a few frames of completely different movies as identical.

that's not really necessary - without going into the ins and outs of how the matching works, the matching algorithm will only match positives, and discard negatives or "similars". For example, if I created a video compilation of 3 different video clips, one after the other and one of those clips was from a video in the fingerprint db, the video compilation would get flagged, because it contained content within that was in the fingerprint db. It is possible for the return to tell you were in the query video (start time/end time) the match occurred, but that's a little more complicated and advanced and would slow down the return response.

How does the algorithm know it's a positive match and not something similar? Well, one ball busting method is once a match is found, the weeds are filtered out from the flowers by testing the match in reverse. Only identical videos will match in both forward and reverse (ie video playing in reverse) - the "similars" will fail the reverse test.

158 views and only 9 votes :disgust

ok so you have a video in the db, and people are uploading 3-5 min clips cut from the members area of that video, and the sponsor approves. would those uploads get flagged?

The content owner could specify if there's any shortened versions of these clips which are not copyrighted, and specify the lengths.

But then, what about trailers that are made up of short clips from multiple copyrighted video sources?

Edit: You could have a whitelist as well as a blacklist. The whitelist would have fingerprints of the promotional material in it, and would be checked against first. If theres a 'strong' match against any of these videos, it would flag as okay and not check the blacklist.

I don't quite follow - what do you mean "the sponsor approves"? The master db is created by content producers, not sponsors. Only verified content producers can add to it. Tube owners can then query this db to see if the uploaded content is allowed or not. If a member has ripped a 5minute clip from a member content video (and that member area video has been licensed to that site owner to show in their member area), then it will get flagged as not approved.

Yes, indeed these ideas need floating which is what I was hoping for, since I don't have all the answers :winkwink:

I like the whitelist/blacklist idea - that would solve a lot of problems. As I mentioned in the content producer's thread, an idea I thought of would be to flag a content with "allow 5 minute clips", but the whitelist would be a better solution, since it is a restrictive approved-of only clips list... anything else, unauthorized.

Great ideas folks :thumbsup

that's what i was asking. so might make your system useless for legal tubes that take legit uploads then?

I don't own a tube.

With that in mind I think it's a great idea, but I think it also puts the tube owner at a competitive disadvantage. Those who don't opt-in, have more (better?) content.

Just for thought, on a forum I run it has a spam service you can connect to, and based on the code returned (0, 1, 2, 3, 4) you configure how you want it to respond.

Code 0 means that the request could not be processed by the spam service, due to an out of date license key or other technical issue.

Code 1 is sent if the spam service determines that the account is unlikely to be spam.

Code 2 is sent by the service if the account is possibly spam

Code 3 is sent by the service if the account is likely spam

Code 4 is sent by the service if the account is a known spammer

Options are:
- Proceed with registration
- Flag as spammer
- Register account, but mark it banned

Seemed like a pretty good way to deal with a restriction system, whether for spam or blacklist/whitelist, users, etc..

No, I still don't get you! Maybe the confusion here is what a content owner licenses his content to be used for. That isn't this thread, but the other one I made. I don't know, I'm neither a content owner, nor a licensee, nor a tube owner, hence the discussion...

This master db is to be generated by content owners - how that db is generated is for them to decide over in the other thread ie, if they have content they never want on a tube, then that is clear cut. What is grey at the moment is 'approved' clips.

If a content producer licenses his content to people to do how they see fit (members are, promotion clips, affiliate clips etc), then this thread isn't for that. This is not about a site owner that has purchased content preventing that content getting on the tubes - it is higher up the hierachy than that. It is about trying to prevent content producer's content *that they control* getting onto the tubes.

There are *many* sites out there that are run by content producers - they are the ones sick to the back teeth of finding their content on the tubes and what this is trying to deal with. There are also content producers that run those sites that produce promotional clips to upload to tubes to drive traffic to their sites - this is what a 'whitelist' would be about...

Anyone that rips a 5 minute clip from a members area should never be allowed to be able to upload that to a tube, if the original was in the master content owners database....

Hence the poll :winkwink:

"yes I'd do anything to reduce the number of dmca requests"

I know if I owned a tube, I'd frikkin want something to auto-check user-uploads. It wouldn't be a "get out of jail free" card, but it would take off a lot of the burden. Youtube actually uses something like this, and I think, but not sure, in a way that manwin implements - opt-in copyright owners have to pay to have user-uploaded content verified it is copyright-free. Hence no movies appear on youtube, but TV streams do...

but many affiliates do, with the owners permission, to make promo video clips they upload to tubes.

Then simply put the owner of that content should not put it on a db of 'not authorised content'.

Sorry, I didn't answer this second part at the time as I got caught up in something else....

Yes, indeed, but this would be the cms author's job, not mine. I have my own cms in the works, which is interested only in secure streaming and won't touch user-uploads. It is for a different market niche, so things like this would need to be added by those that cater to the tube cms market.

But, first thing's first - if I can get this idea off the ground it will be a giant leap in the right direction, but trying to move this lot is like trying to drive a Prosche out of wet concrete :winkwink:

Right, but you'd be returning the codes so I thought it could be an improvement if you returned more than just a yes or no. Factors like if pixel size matched, byte size matched, fingerprint matched, or whatever kind of variables are worth taking into consideration for the cms or developer to find useful.

This is kind of what k0nr4d was alluding to higher up - trust me, a match is pretty much a match. The only variable that comes into play is query movie length. A match can be made on anything > 10 seconds. But match a 10 second clip to a db of hundreds of thousands of 30minute movies and your false positive rate goes up *ONLY* if the real movie is not in the db. If it's in there, the real movie will *always* come out on top.

When the query clip though is unique and not in the db, and the query clip is small (< 1 min), *then* there is the chance of false positives. It must be said, anything > a few minutes will not generate a false positive (in my tests), but to be 100% sure, and to add a scoring system to the results, I would need a db of true negatives to make the script learn from. For this, the true negative db needs to be at least 1% the size of the positive db. I have a negative dataset, but it's from youtube and so it's pretty pathetic in a real world of pron vids, but the capability is there so yes, I'll look at it. Shit, if needs be, I can just use the tubes db and randomly pick out some to use for training. Suffice it to say, the bigger the negative dataset, the higher the confidence ;)

--edit and to add to that 'suffice it to say', throw a positive into that sea of negatives and the whole thing crumbles (hence why I used youtube for my negative dataset in testing) :(

All stuff for me to worry about, not you guys :winkwink:


--edit x2
To be clear, the only thing being sent is the fingerprint, so only that can match... the fingerprint contains all the info to match along with video metadata.

you can't TOS away legal liabilities (ie anti discrimination, censorship etc).

ask eharmony about that (http://www.reuters.com/article/2007/...22132120070531)


youtube automated process is insured, if a bogus takedown happens, the insurance covers the liablity.


If your solution is not insured, just trust me it will be ok,

then you would have to be a world class moron to give up the 100% safety of safe harbor for this solution.

yes you can..

i can just see the headlines now..

" man awarded millions because his home videos of himself jerking off were not allowed on a popular adult video site"



http://my.images.sweetriders.com/pic...ous+Gideon.jpg

so why did eharmony lose their case then

if you can simply tos away legal liabilities they could have simply put in their tos we don't support same sex dating and they would have won.





the whole arguement of the case is that they DIDN't follow the takedown process.

Which is exactly what using this type of service would be.

The megaupload principals would not be in jail today if they had used something like this. Safe harbor simply isn't anything like 100%. If it were, youtube wouldn't have developed and be using a similar system. We can agree the folks at youtube know what they are doing, right? They are paying the cost to run their system (and the cost to insure it) because checking the "do not post" list protects them.

Come to think of it, has there been a case yet that safe safe harbor, designed for ISPs, applies to tubes at all? A tube site's lawyer would certainly argue that it SHOULD apply, but has any court ruled that it DOES? Youtube doesn't appear to be confident that it does.

How this hurts safe harbour I really don't see to be honest. It's still user uploads, it's still not manually inspected (and if the TOS on a site states that you can only upload if you are the IP owner or have rights to do so, then the site is assuming the users are not violating the terms of service), and you are applying a filter to look for copyrighted content. This could probably even be used as a defense that you made the best effort possible to filter content.

Have you looked at phash?

of course, but that's images...

from DMCA safe harbor (thanks Allison):

The tube software author and tube owner can cover 1A+B

I think this thread (tube users) and the other thread (content owners) cover 2A,

This being a free service covers 2B

and the agreed hosting provider for the service will cover 2C

all bases covered no?

I would agree, this wouldn't open you up to any liability as long as it was automated. If this was actually brought to court it would show a good faith effort to follow the spirit of the law which is the opposite of what Gideon usually argues, that being following the letter of the law but knowingly violating the spirit of it.

But why would a false positive even result in legal action? A user doesn't have a civil right that entitles them to have their uploaded porn video displayed on a privately owned tube. Nothing in the original post talks about taking legal action when a video is flagged. So false positives have no place in this discussion.

This is NOT a censorship issue. Not allowing a user uploaded video to be displayed on a privately owned tube is not a free speech, censorship, or discrimination issue. You can use the TOS to say that anyone agreeing to use the tube understands their video is subject to automated approval before being displayed or not.

There is no liability for false positives and the eHarmony suit has no relevance here. Sexual orientation is a protected class in many states, that's why eHarmony choosing to refuse gays or lesbians was violation of state law and not something they could legally deny via a TOS.

it takes about two seconds to realize what paysite vids come from these days.

lol

its not rocket science... half the vids have urls in em lol

Right :thumbsup And take 100 still frames of those videos, and create a composite overlay and the video blurs away into a dark fuzzy background, leaving the URL/logo prominently visible for logo recognition... been there, done that, too easy :winkwink:

http://borkedcoder.com/images/gfy/logo_detect.png

ps, sorry abbywinters.com, I downloaded your video from a tube for research purposes....(see img above) ;)

Tell that to mininova they obeyed the american DMCA and ignored their own countries process

They lost the safe harbor protection of their country and didn't gain the american version so they were fucked over because they decided NOT TO FOLLOW THE LAW.

Same basic principle here.




If that were true the product liability insurance for this a service would be dirt cheap

Of course if it was total bullshit and there was a potential liability then insurance would kill such an offering.



easy way to prove who is right, try and get product liability insurance for the service

if it dirt cheap your right, if it not your wrong.

That why i asked about insurance and who accept the liability for false positives.

If your correct borked could buy insurance dirt cheap and say i accept all liablities for false positives knowing that the insurance will cover those liablities.

???
Just reading up on them, and I don't see how they ignored their own country's process:


So, this is a torrent that was forced by court to implement fingerprinting into their site, to which they did, but initially refused to scan their back catalogue. Hence they were forced to remove all illegal content, which they did. As a consequence they traffic dropped instantly, because that traffic was after illegal content.

How were they fucked over again?