sort and uniq in Linux CLI - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

Output of combined sort and uniq commands

What is the output of the following command sequence on the file fruits.txt containing:

apple
banana
apple
orange
banana
apple

?

Linux CLI

sort fruits.txt | uniq

A

apple
apple
banana
banana
orange

B

apple
banana
orange

C

apple
banana
apple
orange
banana
apple

D

banana
apple
orange

💡 Hint

Remember that sort orders lines alphabetically and uniq removes adjacent duplicates.

The sort command arranges the lines alphabetically, so the file becomes:
apple
apple
apple
banana
banana
orange. Then uniq removes repeated adjacent lines, leaving one of each unique line in order.

💻 Command Output

intermediate

Effect of uniq without sorting

Given a file colors.txt with content:

red
blue
red
green
blue
red

What is the output of the command uniq colors.txt?

Linux CLI

uniq colors.txt

A

red
blue
red
green
blue
red
red

B

red
blue
green

C

red
blue
red
green
blue

D

red
blue
red
green
blue
red

💡 Hint

uniq only removes adjacent duplicate lines.

Since the file is not sorted, duplicate lines are not adjacent. uniq does not remove non-adjacent duplicates, so the output is the same as the input.

📝 Syntax

advanced

Correct usage of uniq to count duplicates

Which command correctly counts how many times each unique line appears in data.txt?

A sort data.txt | uniq -c

B uniq -c data.txt

C sort -c data.txt | uniq -c

D uniq data.txt | sort -c

💡 Hint

Counting duplicates requires sorting first so duplicates are adjacent.

uniq -c counts adjacent duplicates only. Sorting first groups duplicates together, so sort data.txt | uniq -c correctly counts all duplicates.

🔧 Debug

advanced

Why does this command not remove all duplicates?

A user runs uniq -u data.txt to get unique lines from data.txt but sees duplicates still present. Why?

A Because <code>uniq</code> only removes adjacent duplicates; the file is not sorted.

B Because <code>-u</code> option is invalid and ignored.

C Because <code>uniq</code> removes all duplicates regardless of order.

D Because the file contains hidden characters causing errors.

💡 Hint

Think about how uniq works with adjacent lines.

uniq -u prints only lines that are unique and not repeated adjacent lines. If the file is not sorted, duplicates may not be adjacent, so they remain.

🚀 Application

expert

Extract unique sorted IP addresses from a log

You have a log file access.log with many lines, each starting with an IP address. Which command extracts all unique IP addresses sorted alphabetically?

A sort access.log | cut -d' ' -f1 | uniq

B uniq access.log | cut -d' ' -f1 | sort

C cut -d' ' -f1 access.log | sort | uniq

D cut -d' ' -f1 access.log | uniq | sort

💡 Hint

Extract the first field, then sort, then remove duplicates.

First, cut -d' ' -f1 extracts the IP addresses. Then sort arranges them alphabetically, grouping duplicates. Finally, uniq removes duplicates.

Practice

1. What does the uniq command do when used alone on a text file?

easy

A. Deletes all empty lines

B. Removes repeated adjacent lines

C. Counts all lines in the file

D. Sorts all lines alphabetically

Solution

1. Step 1: Understand the uniq command behavior

   The uniq command filters out repeated lines only if they are next to each other.

2. Step 2: Compare with other options

   It does not sort lines or count all lines; it only removes adjacent duplicates.

3. Final Answer:

   Removes repeated adjacent lines -> Option B

4. Quick Check:

   uniq removes adjacent duplicates = D [OK]

Hint: uniq only removes duplicates if they are next to each other [OK]

Common Mistakes:

• Thinking uniq sorts lines
• Assuming uniq removes all duplicates anywhere
• Confusing uniq with wc (word count)

2. Which of the following is the correct syntax using a pipe to sort a file named data.txt and remove duplicate lines?

easy

A. sort data.txt | uniq

B. uniq data.txt | sort

C. sort -u data.txt

D. uniq -s data.txt

Solution

1. Step 1: Understand the order of commands

   To remove duplicates correctly, lines must be sorted first so duplicates become adjacent.

2. Step 2: Check each option

   sort data.txt | uniq sorts then pipes to uniq, which removes adjacent duplicates. uniq data.txt | sort runs uniq first, which won't remove all duplicates if unsorted. sort -u data.txt uses sort -u which is valid but not the pipe syntax asked. uniq -s data.txt is invalid syntax.

3. Final Answer:

   sort data.txt | uniq -> Option A

4. Quick Check:

   Sort before uniq to remove duplicates = A [OK]

Hint: Always sort before uniq to remove duplicates correctly [OK]

Common Mistakes:

• Running uniq before sort
• Using invalid uniq options
• Confusing sort -u with uniq

3. Given a file names.txt with contents:

bob
alice
bob
carol
alice
bob

What is the output of sort names.txt | uniq?

medium

A. alice
bob
carol

B. bob
alice
carol

C. alice
bob
bob
carol
alice
bob

D. bob
carol
alice

Solution

1. Step 1: Sort the file lines

   Sorting the lines alphabetically gives: alice, alice, bob, bob, bob, carol.

2. Step 2: Apply uniq to remove adjacent duplicates

   uniq removes repeated adjacent lines, so only one of each remains: alice, bob, carol.

3. Final Answer:

   alice bob carol -> Option A

4. Quick Check:

   Sort then uniq removes duplicates = alice, bob, carol [OK]

Hint: Sort first, then uniq removes duplicates in order [OK]

Common Mistakes:

• Not sorting before uniq
• Expecting uniq to remove all duplicates anywhere
• Confusing output order

4. You run sort data.txt | uniq but notice duplicates remain. What is the most likely cause?

medium

A. sort command does not work on text files

B. uniq command is misspelled

C. The file contains lines with trailing spaces or tabs

D. The file is empty

Solution

1. Step 1: Understand how sort and uniq treat whitespace

   sort and uniq treat lines with trailing spaces or tabs as different lines, so duplicates with extra spaces won't be adjacent identical.

2. Step 2: Check other options

   uniq misspelling would cause error, not duplicates remaining. sort works on text files. Empty file means no duplicates.

3. Final Answer:

   The file contains lines with trailing spaces or tabs -> Option C

4. Quick Check:

   Trailing spaces cause uniq to miss duplicates = C [OK]

Hint: Check for trailing spaces if duplicates remain after sort|uniq [OK]

Common Mistakes:

• Assuming uniq always removes duplicates regardless of spaces
• Ignoring trailing whitespace differences
• Thinking sort fails on text files

5. You want to count how many unique IP addresses appear in access.log. Which command correctly sorts, removes duplicates, and counts unique lines?

hard

A. sort access.log | uniq -c

B. uniq access.log | wc -l

C. uniq -c access.log | sort

D. sort -u access.log | wc -l

Solution

1. Step 1: Understand the goal

   You want the count of unique IP addresses, so you need to remove duplicates and count the unique lines.

2. Step 2: Analyze each option

   sort access.log | uniq -c counts occurrences but does not sum total unique lines. uniq -c access.log | sort counts duplicates before sorting, so misses some. sort -u access.log | wc -l uses sort -u to get unique lines, then counts lines with wc -l, which gives the unique count. uniq access.log | wc -l counts lines after uniq without sorting, so duplicates may remain.

3. Final Answer:

   sort -u access.log | wc -l -> Option D

4. Quick Check:

   sort -u then wc -l counts unique lines = B [OK]

Hint: Use sort -u then wc -l to count unique lines fast [OK]

Common Mistakes:

• Using uniq -c without sorting first
• Counting lines before removing duplicates
• Confusing uniq -c output with total unique count