Article Categories
- All Categories
-
Data Structure
-
Networking
-
RDBMS
-
Operating System
-
Java
-
MS Excel
-
iOS
-
HTML
-
CSS
-
Android
-
Python
-
C Programming
-
C++
-
C#
-
MongoDB
-
MySQL
-
Javascript
-
PHP
How to removes duplicate lines from a sorted file in Linux?
To remove duplicate lines from a sorted file and make it unique, we use the uniq command in the Linux system. The uniq command work as a kind of filter program that reports out the duplicate lines in a file. It filters adjacent matching lines from the input and gives a unique output. This command is also available in the Windows and IBM i operating system.
Syntax
The general syntax of the uniq command is as follows
uniq [OPTION]... [INPUT [OUTPUT]]
Brief description of options available in the fmt command.
| Sr.No. | Option & Description |
|---|---|
| 1 |
-c, --count Display how many times line was repeated. |
| 2 |
-d—repeated Display only repeated lines, one for each group. |
| 3 |
-D Display all duplicate lines. |
| 4 |
-f, --skip-fields=N Avoid comparing the first N fields. |
| 5 |
-i, --ignore-case While comparing ignore differences in case. |
| 6 |
-s, --skip-chars=N Avoid comparing the first N characters. |
| 7 |
-u, --unique Prints only unique lines |
| 8 |
-w, --check-chars=N Line delimiter is NULL, not newline |
| 9 |
-v, --verbose Compare no more than N characters in lines. |
| 10 |
--help Display help and exit. |
| 11 |
--version Output the version information and exit. |
To print after removal repeated lines in a file, we use the uniq command in the Linux system as shown below.
$ cat >text.txt Print only unique lines. The earth is round. The earth is round. Welcome to the tutorialpoint... Welcome to the tutorialspint... $ uniq text.txt Print only unique lines. The earth is round. Welcome to the tutorialpoint...
To print number of duplicate lines of a file, we use the -c or --count option with the unique command as shown below.
$ uniq –c text.txt 2 The earth is round. 2 Welcome to the tutorialspoint... 1 Print only unique lines.
To prints only unique lines of a file, we use the -u or –unique option with the uniq command as shown below.
$ uniq –u text.txt Print only unique lines.
To check more information about the uniq command, we use the --help option with the uniq command in the Linux operating system as shown below.
$ uniq --help
To check version information of the uniq command, we use the --version option with the uniq command in the Linux operating system as shown below.
$ uniq --version
7K+ Views
