Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
How to removes duplicate lines from a sorted file in Linux?
To remove duplicate lines from a sorted file and make it unique, we use the uniq command in the Linux system. The uniq command work as a kind of filter program that reports out the duplicate lines in a file. It filters adjacent matching lines from the input and gives a unique output. This command is also available in the Windows and IBM i operating system.
Syntax
The general syntax of the uniq command is as follows
uniq [OPTION]... [INPUT [OUTPUT]]
Brief description of options available in the fmt command.
| Sr.No. | Option & Description |
|---|---|
| 1 | -c, --count Display how many times line was repeated. |
| 2 | -d—repeated Display only repeated lines, one for each group. |
| 3 | -D Display all duplicate lines. |
| 4 | -f, --skip-fields=N Avoid comparing the first N fields. |
| 5 | -i, --ignore-case While comparing ignore differences in case. |
| 6 | -s, --skip-chars=N Avoid comparing the first N characters. |
| 7 | -u, --unique Prints only unique lines |
| 8 | -w, --check-chars=N Line delimiter is NULL, not newline |
| 9 | -v, --verbose Compare no more than N characters in lines. |
| 10 | --help Display help and exit. |
| 11 | --version Output the version information and exit. |
To print after removal repeated lines in a file, we use the uniq command in the Linux system as shown below.
$ cat >text.txt Print only unique lines. The earth is round. The earth is round. Welcome to the tutorialpoint... Welcome to the tutorialspint... $ uniq text.txt Print only unique lines. The earth is round. Welcome to the tutorialpoint...
To print number of duplicate lines of a file, we use the -c or --count option with the unique command as shown below.
$ uniq –c text.txt 2 The earth is round. 2 Welcome to the tutorialspoint... 1 Print only unique lines.
To prints only unique lines of a file, we use the -u or –unique option with the uniq command as shown below.
$ uniq –u text.txt Print only unique lines.
To check more information about the uniq command, we use the --help option with the uniq command in the Linux operating system as shown below.
$ uniq --help
To check version information of the uniq command, we use the --version option with the uniq command in the Linux operating system as shown below.
$ uniq --version
7K+ Views
