removing duplicate lines... - GoFuckYourself.com

qwe · 06-22-2008, 07:03 PM

is there a program that can remove duplicate lines in .txt file ? lines might contain things like ;.,- with some letters/digits

just a punk · 06-22-2008, 07:29 PM

Quote:

Originally Posted by qwe

is there a program that can remove duplicate lines in .txt file ? lines might contain things like ;.,- with some letters/digits

Here is what you need: http://www.maturedesign.com/junk/gfy/removedupes.exe

It's a command-line application. So you should use it as follows:

removedupes.exe infile.txt outfile.txt

So if infile.txt was containing something like this:
0000
123
234
;wer;
!@#l
!@#!23kl
123
23423
234234
qeo23
234
234234
22392jwd89
K!@#L!@(#12
!@#!23kl
!@#l
123

the outfile.txt will be the following:
!@#!23kl
!@#l
;wer;
0000
123
22392jwd89
234
23423
234234
K!@#L!@(#12
qeo23

P.S. Don't worry about viruses/trojans etc. I wrote this utility myself just a couple of minutes ago.

fris · 06-22-2008, 07:31 PM

http://www.rightscripts.com/listrearrange/

AlienQ - BANNED FOR LIFE · 06-22-2008, 07:31 PM

Globals are your friend.

Voodoo · 06-22-2008, 07:56 PM

http://www.flashpeak.com/slimlist/slimlist.htm

qwe · 06-22-2008, 07:57 PM

Quote:

Originally Posted by cyberxxx

Here is what you need: http://www.maturedesign.com/junk/gfy/removedupes.exe

It's a command-line application. So you should use it as follows:

removedupes.exe infile.txt outfile.txt

So if infile.txt was containing something like this:
0000
123
234
;wer;
!@#l
!@#!23kl
123
23423
234234
qeo23
234
234234
22392jwd89
K!@#L!@(#12
!@#!23kl
!@#l
123

the outfile.txt will be the following:
!@#!23kl
!@#l
;wer;
0000
123
22392jwd89
234
23423
234234
K!@#L!@(#12
qeo23

P.S. Don't worry about viruses/trojans etc. I wrote this utility myself just a couple of minutes ago.

awsome thx

qwe · 06-22-2008, 08:01 PM

Quote:

Originally Posted by Voodoo

http://www.flashpeak.com/slimlist/slimlist.htm

awsome, thx

rogerrabbit00 · 06-22-2008, 10:25 PM

downloads.com Search "text pad" U install a BS exe you are hacked!

just a punk · 06-23-2008, 04:05 AM

Quote:

Originally Posted by rogerrabbit00

downloads.com Search "text pad" U install a BS exe you are hacked!

Isn't "text pad" an exe file? Do you mean everyone who installed text pad or any other application is hacked?

abshard · 06-23-2008, 04:39 AM

What i use

http://www.rlvision.com/dupli/about.asp

CunningStunt · 06-23-2008, 04:49 AM

Copy the info into Excel, download asap utilities, and go from there. has masses of very good data cleaning utlities in there, even muppets like me can use.

06-22-2008, 07:31 PM	#3
fris Too lazy to set a custom title Industry Role: Join Date: Aug 2002 Posts: 55,372	http://www.rightscripts.com/listrearrange/ __________________ Since 1999: 69 Adult Industry awards for Best Hosting Company and professional excellence. WP Stuff

06-22-2008, 07:31 PM	#4
AlienQ - BANNED FOR LIFE best designer on GFY Join Date: Mar 2003 Location: IALIEN.COM - High Definition Video and Photographic Productions -ICQ 78943384 Posts: 30,307	Globals are your friend. __________________ NAKED HOSTING FTW!11 I'm On The INSANE PLAN $9.95/mo! \| The Alien Blog Adult News Worth Reading Updated Daily \| Content For Sale! 641 PICS 216 MINUTES OF VIDEO $350.00 \|ICQ: 78943384 \|

06-22-2008, 07:56 PM	#5
Voodoo ♥ ♦ ♣ ♠ Industry Role: Join Date: Sep 2002 Posts: 10,591	http://www.flashpeak.com/slimlist/slimlist.htm __________________ "I'm selflessly supporting the common good, but only coincidentally looking out for No.1."

06-22-2008, 10:25 PM	#8
rogerrabbit00 Confirmed User Industry Role: Join Date: Aug 2006 Location: Costa Rica Posts: 315	downloads.com Search "text pad" U install a BS exe you are hacked! __________________ Dating Email Traffic for Sale. www.SplitTraffic.com Skype: Rogerwilson00

06-22-2008, 07:03 PM	#1
qwe Confirmed User Join Date: Jul 2003 Posts: 2,109	removing duplicate lines... is there a program that can remove duplicate lines in .txt file ? lines might contain things like ;.,- with some letters/digits

06-23-2008, 04:39 AM	#10
abshard Confirmed User Industry Role: Join Date: Jan 2002 Location: AZ Posts: 6,370	What i use http://www.rlvision.com/dupli/about.asp

06-23-2008, 04:49 AM	#11
CunningStunt Confirmed User Industry Role: Join Date: Aug 2006 Posts: 5,594	Copy the info into Excel, download asap utilities, and go from there. has masses of very good data cleaning utlities in there, even muppets like me can use.