Removes duplicates from a text file

[Joe Caverly]

     _x64: 1
   _admin: 1
_elevated: 1

TCC  26.02.42 x64   Windows 10 [Version 10.0.18363.1082]
BuildNumber  Caption                   CSDVersion  OSArchitecture  Version
18363        Microsoft Windows 10 Pro              64-bit          10.0.18363

e:\utils>which sort.exe
sort.exe is an external : C:\WINDOWS\system32\sort.exe

I can use SORT.EXE to sort a text file;

e:\utils>sort unique.txt
123
132
213
231
312
312
321

If I want to remove duplicates from the sorted text file;

e:\utils>sort /uniq unique.txt
123
132
213
231
312
321

/uniq appears to be an un-documented switch for SORT.EXE on my Windows 10 system.

Not sure if this available on Windows 7 or not.

Joe

 

[drwtsn32]

Interesting find! The /uniq switch doesn't work on Windows 7's sort.exe.

You can also achieve this quite easily in PowerShell.

 

[Joe Caverly]

e:\utils>pshell /s "type unique.txt | sort -unique"
123
132
213
231
312
321

Joe

 

[Charles G]

tpipe /dup=0,0,1,65535,0,""

will work too