removing duplicate lines...
 

is there a program that can remove duplicate lines in .txt file ? lines might contain things like ;.,- with some letters/digits

Here is what you need: http://www.maturedesign.com/junk/gfy/removedupes.exe

It's a command-line application. So you should use it as follows:

removedupes.exe infile.txt outfile.txt

So if infile.txt was containing something like this:
0000
123
234
;wer;
!@#l
!@#!23kl
123
23423
234234
qeo23
234
234234
22392jwd89
K!@#L!@(#12
!@#!23kl
!@#l
123

the outfile.txt will be the following:
!@#!23kl
!@#l
;wer;
0000
123
22392jwd89
234
23423
234234
K!@#L!@(#12
qeo23

P.S. Don't worry about viruses/trojans etc. I wrote this utility myself just a couple of minutes ago.

Globals are your friend.

awsome thx :thumbsup

awsome, thx

downloads.com Search "text pad" U install a BS exe you are hacked!

Isn't "text pad" an exe file? Do you mean everyone who installed text pad or any other application is hacked? :1orglaugh

What i use

http://www.rlvision.com/dupli/about.asp

Copy the info into Excel, download asap utilities, and go from there. has masses of very good data cleaning utlities in there, even muppets like me can use.