How to Use uniq Command in Bash: Syntax and Examples

In bash, uniq filters out or reports repeated adjacent lines in a file or input stream. Use uniq filename to remove duplicate consecutive lines, or combine with sort to handle all duplicates. Options like -c count occurrences, and -d shows only duplicates.

Syntax

The basic syntax of uniq is:

• uniq [options] [input_file] [output_file]

Where:

• input_file is the file to read (or standard input if omitted)
• output_file is where to write results (or standard output if omitted)
• options modify behavior, like counting or showing duplicates only

bash

uniq [options] [input_file] [output_file]

Example

This example shows how to remove duplicate adjacent lines from a file and how to count occurrences of each line.

bash

echo -e "apple\napple\nbanana\nbanana\nbanana\ncherry" > fruits.txt
uniq fruits.txt
uniq -c fruits.txt

Output

apple
banana
cherry
      2 apple
      3 banana
      1 cherry

Common Pitfalls

uniq only removes or detects adjacent duplicate lines. If duplicates are scattered, uniq alone won't remove them. You usually need to sort the input first.

Also, forgetting to use options like -c or -d can lead to unexpected output.

bash

echo -e "banana\napple\nbanana\napple" > mixed.txt
uniq mixed.txt
sort mixed.txt | uniq
sort mixed.txt | uniq -c

Output

banana
apple
banana
apple
apple
apple
banana
banana
      2 apple
      2 banana

Quick Reference

Option	Description
-c	Prefix lines by the number of occurrences
-d	Only print duplicate lines
-u	Only print unique lines
-i	Ignore case when comparing
-f N	Skip first N fields
-s N	Skip first N characters
-w N	Compare no more than N characters

Key Takeaways

uniq filters or counts only adjacent duplicate lines in bash.

Use sort before uniq to handle all duplicates in a file.

The -c option counts occurrences; -d shows duplicates only.

Without sorting, uniq won't remove non-adjacent duplicates.

Combine uniq options to customize output for your needs.