How to removes duplicate lines from a sorted file in Linux?

To remove duplicate lines from a sorted file and make it unique, we use the uniq command in the Linux system. The uniq command work as a kind of filter program that reports out the duplicate lines in a file. It filters adjacent matching lines from the input and gives a unique output. This command is also available in the Windows and IBM i operating system.

Syntax

The general syntax of the uniq command is as follows

uniq [OPTION]... [INPUT [OUTPUT]]

Brief description of options available in the fmt command.

To print after removal repeated lines in a file, we use the uniq command in the Linux system as shown below.

$ cat >text.txt
Print only unique lines.
The earth is round.
The earth is round.
Welcome to the tutorialpoint...
Welcome to the tutorialspint...
$ uniq text.txt
Print only unique lines.
The earth is round.
Welcome to the tutorialpoint...

To print number of duplicate lines of a file, we use the -c or --count option with the unique command as shown below.

$ uniq –c text.txt
2 The earth is round.
2 Welcome to the tutorialspoint...
1 Print only unique lines.

To prints only unique lines of a file, we use the -u or –unique option with the uniq command as shown below.

$ uniq –u text.txt
Print only unique lines.

To check more information about the uniq command, we use the --help option with the uniq command in the Linux operating system as shown below.

$ uniq --help

To check version information of the uniq command, we use the --version option with the uniq command in the Linux operating system as shown below.

$ uniq --version