Anyone know a great dup text checker and/or a comprehensive algorithm for it...
 

Been using dupecop.com which is pretty good but I suspect (like others that actually SAY they do), they may be saving the text which I don't want. So I wrote my own using the the shingles algorithm but I need more details on whether or not I should be using 2 or 3 words, whether stop words should be removed etc. etc. etc... My results can vary quite a bit from dupe cop.

text pad if you are trying to just de dupe...

Oh, hello.

Not even close to what I'm looking for.. that removes duplicate lines from a file. I'm looking for code that compares too sets of text and gives a "ranking" as to how unique the 2nd set of text is compared to the first set.

Anyone???