GoFuckYourself.com - Adult Webmaster Forum - View Single Post - UNIX question -- how to sort and remove duplicates?

TheFLY · 07-25-2002, 10:39 AM

I have a huge file that's full of tons of duplicate lines -- I want to remove all the duplicate lines from the file.

I did a "man sort" and I didn't see any option to remove/ignore duplicate lines --

Anyone know how to do it? I'm sure it's easy...

07-25-2002, 10:39 AM
TheFLY So Fucking Banned Join Date: Jan 2001 Location: http://www.thefly.net/ --- Quit your job and live off steady traffic. Posts: 11,856	UNIX question -- how to sort and remove duplicates? I have a huge file that's full of tons of duplicate lines -- I want to remove all the duplicate lines from the file. I did a "man sort" and I didn't see any option to remove/ignore duplicate lines -- Anyone know how to do it? I'm sure it's easy...