sort and uniq in pipelines in Bash Scripting - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

What is the output of this pipeline?

Consider a file fruits.txt with the following lines:

apple
banana
apple
orange
banana
banana

What is the output of this command?

cat fruits.txt | sort | uniq

Bash Scripting

cat fruits.txt | sort | uniq

A

apple
banana
banana
orange

B

apple
apple
banana
banana
banana
orange

C

apple
banana
orange

D

banana
apple
orange

💡 Hint

Think about what sort and uniq do when combined.

The sort command arranges lines alphabetically. Then uniq removes consecutive duplicate lines. So duplicates are removed after sorting, leaving one of each fruit.

💻 Command Output

intermediate

What does this pipeline output?

Given a file colors.txt with:

red
blue
red
green
blue
blue

What is the output of:

cat colors.txt | uniq | sort

Bash Scripting

cat colors.txt | uniq | sort

A

blue
blue
green
red
red

B

red
blue
green

C

red
blue
red
green
blue
blue

D

red
green
blue

💡 Hint

Remember that uniq only removes adjacent duplicates.

The uniq command removes only consecutive duplicates, so the first uniq keeps the first 'red' and removes the second only if adjacent. Then sort arranges lines alphabetically.

📝 Syntax

advanced

Which pipeline correctly counts unique lines ignoring case?

You want to count how many unique lines are in data.txt, ignoring case differences. Which command pipeline does this correctly?

A cat data.txt | sort -f | uniq -i | wc -l

B cat data.txt | sort | uniq -i | wc -l

C cat data.txt | uniq -i | sort | wc -l

D cat data.txt | sort -f | uniq | wc -l

💡 Hint

Check which commands support case-insensitive options and how they affect sorting and uniqueness.

sort -f sorts ignoring case, grouping lines that match ignoring case together. uniq -i then removes consecutive duplicates ignoring case. wc -l counts the remaining unique lines ignoring case.

🔧 Debug

advanced

Why does this pipeline not remove all duplicates?

You run this command:

cat list.txt | uniq | sort

But you notice duplicates remain in the output. Why?

A Because <code>cat</code> removes duplicates before piping.

B Because <code>uniq</code> only removes adjacent duplicates, and sorting after <code>uniq</code> does not group duplicates together.

C Because <code>uniq</code> requires the <code>-d</code> option to remove duplicates.

D Because <code>sort</code> does not work on piped input.

💡 Hint

Think about the order of commands and how uniq works.

uniq only removes duplicates if they are next to each other. Since sorting happens after uniq, duplicates may be separated and not removed.

🚀 Application

expert

How to find the top 3 most frequent words in a file?

You have a file words.txt with many words, one per line. You want to find the top 3 most frequent words along with their counts. Which pipeline achieves this?

A cat words.txt | sort | uniq | sort -nr | head -3

B cat words.txt | uniq -c | sort -nr | head -3

C cat words.txt | sort -nr | uniq -c | head -3

D cat words.txt | sort | uniq -c | sort -nr | head -3

💡 Hint

Count duplicates after sorting, then sort counts numerically in reverse order.

First, sort groups identical words together. Then uniq -c counts occurrences. Next, sort -nr sorts by count descending. Finally, head -3 shows top 3.

Practice

1. What is the main purpose of using sort | uniq together in a bash pipeline?

easy

A. To get a sorted list with duplicates removed

B. To count the number of lines in a file

C. To sort lines but keep duplicates

D. To find the longest line in a file

Solution

1. Step 1: Understand what sort does

   The sort command arranges lines in alphabetical or numerical order.

2. Step 2: Understand what uniq does after sorting

   The uniq command removes duplicate adjacent lines, so sorting first groups duplicates together for removal.

3. Final Answer:

   To get a sorted list with duplicates removed -> Option A

4. Quick Check:

   sort + uniq = sorted unique lines [OK]

Hint: Always sort before uniq to remove all duplicates [OK]

Common Mistakes:

• Using uniq before sort, which misses duplicates
• Thinking uniq counts lines instead of removing duplicates
• Assuming sort removes duplicates by itself

2. Which of the following is the correct syntax to remove duplicate lines from file.txt and sort the output?

easy

A. uniq file.txt | sort

B. sort file.txt | uniq

C. sort | uniq file.txt

D. uniq | sort file.txt

Solution

1. Step 1: Identify correct command order

   The file must be sorted first, so sort file.txt comes before uniq.

2. Step 2: Check syntax correctness

   sort file.txt | uniq correctly pipes sorted output to uniq for duplicate removal.

3. Final Answer:

   sort file.txt | uniq -> Option B

4. Quick Check:

   sort first, then uniq = correct syntax [OK]

Hint: Pipe sorted output into uniq: sort file | uniq [OK]

Common Mistakes:

• Placing uniq before sort, which fails to remove all duplicates
• Incorrect pipe placement causing syntax errors
• Trying to pass file as argument to uniq instead of sort

3. Given a file names.txt with contents:

bob
alice
bob
carol
alice
bob

What is the output of sort names.txt | uniq?

medium

A. alice
bob
bob
carol
alice
bob

B. bob
alice
carol

C. bob
carol
alice

D. alice
bob
carol

Solution

1. Step 1: Sort the file contents

   Sorting lines alphabetically gives: alice, alice, bob, bob, bob, carol.

2. Step 2: Apply uniq to remove duplicates

   Uniq removes repeated adjacent lines, resulting in: alice, bob, carol.

3. Final Answer:

   alice\nbob\ncarol -> Option D

4. Quick Check:

   sort + uniq = unique sorted lines [OK]

Hint: Sort first to group duplicates, then uniq removes repeats [OK]

Common Mistakes:

• Not sorting first, so uniq misses duplicates
• Expecting original order instead of sorted
• Confusing uniq output with counting duplicates

4. You run uniq file.txt | sort to remove duplicates and sort lines, but duplicates remain. What is the likely problem?

medium

A. uniq only removes adjacent duplicates, so sorting after uniq misses duplicates

B. sort command is not working without options

C. file.txt is empty, so no output

D. uniq requires -c option to remove duplicates

Solution

1. Step 1: Understand uniq behavior

   Uniq removes only adjacent duplicate lines, so if duplicates are scattered, it won't remove them.

2. Step 2: Analyze command order effect

   Sorting after uniq means duplicates are not grouped before uniq runs, so duplicates remain.

3. Final Answer:

   uniq only removes adjacent duplicates, so sorting after uniq misses duplicates -> Option A

4. Quick Check:

   uniq needs sorted input to remove all duplicates [OK]

Hint: Sort before uniq to group duplicates for removal [OK]

Common Mistakes:

• Thinking uniq removes all duplicates regardless of order
• Believing sort alone removes duplicates
• Using uniq -c instead of uniq for removal

5. You have a log file access.log with many repeated IP addresses scattered randomly. You want a sorted list of unique IPs. Which command pipeline correctly achieves this?

hard

A. cat access.log | uniq | sort

B. uniq access.log | sort

C. sort -u access.log

D. uniq access.log | sort -u

Solution

1. Step 1: Understand sort -u option

   sort -u sorts and removes duplicates in one step efficiently.

2. Step 2: Why others fail

   Options A, B, and D all apply uniq before sorting, missing non-adjacent duplicates since IPs are scattered randomly.

3. Final Answer:

   sort -u access.log -> Option C

4. Quick Check:

   sort -u = sorted unique lines in one command [OK]

Hint: Use sort -u for sorted unique lines in one step [OK]

Common Mistakes:

• Using uniq before sort, missing duplicates
• Not knowing sort -u combines sorting and uniqueness
• Using unnecessary cat command