Removing duplicates (drop_duplicates) in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of drop_duplicates with subset

What is the output DataFrame after running this code?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({
    'A': [1, 2, 2, 3, 1],
    'B': ['x', 'y', 'y', 'z', 'x'],
    'C': [10, 20, 20, 30, 40]
})

result = df.drop_duplicates(subset=['A'])
print(result)

A

   A  B   C
0  1  x  10
1  2  y  20
2  2  y  20
3  3  z  30
4  1  x  40

B

   A  B   C
0  1  x  10
2  2  y  20
3  3  z  30

C

   A  B   C
1  2  y  20
2  2  y  20
3  3  z  30

D

   A  B   C
0  1  x  10
1  2  y  20
3  3  z  30

💡 Hint

Look at the first occurrence of each unique value in column 'A'.

drop_duplicates with subset=['A'] keeps the first row for each unique value in column 'A'. Rows with duplicate 'A' values after the first are removed.

❓ data_output

intermediate

Number of rows after drop_duplicates

How many rows remain after removing duplicates based on columns 'A' and 'B'?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({
    'A': [1, 2, 2, 3, 1],
    'B': ['x', 'y', 'y', 'z', 'x'],
    'C': [10, 20, 20, 30, 40]
})

result = df.drop_duplicates(subset=['A', 'B'])
print(len(result))

A 3

B 4

C 5

D 2

💡 Hint

Check unique pairs of (A, B) in the DataFrame.

The unique pairs are (1,x), (2,y), (3,z). The pair (1,x) appears twice but only first is kept. So total unique pairs are 3, but since duplicates are removed, rows with indices 0,1,3 remain. The row with index 4 is duplicate of (1,x) so removed. So total rows after drop_duplicates is 3.

🔧 Debug

advanced

Error when using drop_duplicates with inplace=True

What error will this code raise?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({'A': [1, 1, 2], 'B': [3, 3, 4]})
df.drop_duplicates(inplace=True, subset=['A', 'B'])
print(df)

A NameError: name 'df' is not defined

B No error, prints the DataFrame with duplicates removed

C AttributeError: 'NoneType' object has no attribute 'print'

D TypeError: drop_duplicates() got an unexpected keyword argument 'inplace'

💡 Hint

Check what drop_duplicates returns when inplace=True.

drop_duplicates with inplace=True modifies the DataFrame in place and returns None. The variable df remains defined and prints correctly.

🚀 Application

advanced

Removing duplicates but keeping last occurrence

Which option correctly removes duplicates from DataFrame df based on column 'A' but keeps the last occurrence?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({
    'A': [1, 2, 2, 3, 1],
    'B': ['x', 'y', 'y', 'z', 'x'],
    'C': [10, 20, 20, 30, 40]
})

A df.drop_duplicates(subset=['A'], keep='none')

B df.drop_duplicates(subset=['A'], keep='first')

C df.drop_duplicates(subset=['A'], keep='last')

D df.drop_duplicates(subset=['A'], keep=False)

💡 Hint

The keep parameter controls which duplicate to keep.

keep='last' keeps the last occurrence of each duplicate, removing earlier ones.

🧠 Conceptual

expert

Effect of drop_duplicates on index

After using drop_duplicates on a DataFrame, what happens to the index by default?

A The original index values are preserved, including gaps from removed rows

B The index is reset to a new continuous range starting from 0

C The index is dropped and replaced with a default integer index without gaps

D The index is converted to a MultiIndex based on duplicate columns

💡 Hint

Think about whether drop_duplicates changes the index or just drops rows.

drop_duplicates removes duplicate rows but keeps the original index values, so gaps remain unless reset_index() is called.

Practice

1. What does the drop_duplicates() function do in a pandas DataFrame?

easy

A. Fills missing values with zeros

B. Removes repeated rows based on all columns by default

C. Sorts the DataFrame rows in ascending order

D. Changes the data type of columns

Solution

1. Step 1: Understand the function purpose

   drop_duplicates() is used to remove repeated rows in a DataFrame.

2. Step 2: Default behavior

   By default, it checks all columns to find duplicates and keeps the first occurrence.

3. Final Answer:

   Removes repeated rows based on all columns by default -> Option B

4. Quick Check:

   drop_duplicates() removes repeated rows [OK]

Hint: Remember: drop_duplicates removes repeated rows [OK]

Common Mistakes:

• Thinking it sorts data
• Confusing with fillna()
• Assuming it changes data types

2. Which of the following is the correct syntax to remove duplicate rows from a DataFrame df?

easy

A. df.drop_duplicates[]

B. df.remove_duplicates()

C. df.drop_duplicates()

D. df.drop_duplicates{}

Solution

1. Step 1: Identify correct method call

   The method to remove duplicates is drop_duplicates(), called with parentheses.

2. Step 2: Check syntax correctness

   Only df.drop_duplicates() is valid syntax; brackets or braces are incorrect.

3. Final Answer:

   df.drop_duplicates() -> Option C

4. Quick Check:

   Use parentheses to call methods [OK]

Hint: Use parentheses () to call drop_duplicates method [OK]

Common Mistakes:

• Using square brackets instead of parentheses
• Using a non-existent method name
• Using curly braces instead of parentheses

3. Given the DataFrame df below, what will df.drop_duplicates(subset=['Name']) return?

import pandas as pd
data = {'Name': ['Anna', 'Bob', 'Anna', 'Cody'], 'Age': [25, 30, 25, 22]}
df = pd.DataFrame(data)

medium

A. [{'Name': 'Anna', 'Age': 25}, {'Name': 'Bob', 'Age': 30}, {'Name': 'Cody', 'Age': 22}]

B. [{'Name': 'Anna', 'Age': 25}, {'Name': 'Anna', 'Age': 25}, {'Name': 'Bob', 'Age': 30}, {'Name': 'Cody', 'Age': 22}]

C. [{'Name': 'Bob', 'Age': 30}, {'Name': 'Cody', 'Age': 22}]

D. [{'Name': 'Anna', 'Age': 25}, {'Name': 'Bob', 'Age': 30}]

Solution

1. Step 1: Understand subset parameter

   Using subset=['Name'] means duplicates are checked only on the 'Name' column.

2. Step 2: Identify duplicates by 'Name'

   'Anna' appears twice; only the first occurrence is kept, others dropped.

3. Final Answer:

   [{'Name': 'Anna', 'Age': 25}, {'Name': 'Bob', 'Age': 30}, {'Name': 'Cody', 'Age': 22}] -> Option A

4. Quick Check:

   Duplicates dropped by 'Name' column [OK]

Hint: Use subset to check duplicates on specific columns [OK]

Common Mistakes:

• Not using subset and dropping all duplicates
• Keeping all duplicates instead of first
• Confusing which rows are dropped

4. What is wrong with this code snippet to remove duplicates from DataFrame df?

df.drop_duplicates(inplace=True)

After running it, duplicates still appear in df. Why?

medium

A. The DataFrame has no duplicates to remove

B. drop_duplicates() does not modify the original DataFrame even with inplace=True

C. The DataFrame was not reassigned and inplace=True is ignored

D. The inplace parameter is misspelled

Solution

1. Step 1: Check inplace parameter usage

   inplace=True modifies the original DataFrame, so no reassignment needed.

2. Step 2: Consider DataFrame content

   If duplicates still appear, likely the DataFrame had no duplicates to remove.

3. Final Answer:

   The DataFrame has no duplicates to remove -> Option A

4. Quick Check:

   inplace=True modifies original; no duplicates means no change [OK]

Hint: Check if DataFrame actually has duplicates before drop_duplicates [OK]

Common Mistakes:

• Assuming inplace=True does nothing
• Thinking drop_duplicates always changes data
• Not verifying if duplicates exist

5. You have a DataFrame df with columns ['ID', 'Name', 'Score']. Some rows have duplicate ID values but different Score. How do you keep only the row with the highest Score for each ID?

hard

A. Use df.drop_duplicates(subset=['Score'])

B. Use df.drop_duplicates(subset=['ID']) without sorting

C. Use df.drop_duplicates(keep='last') without sorting

D. Use df.sort_values('Score', ascending=False).drop_duplicates(subset=['ID'])

Solution

1. Step 1: Sort DataFrame by Score descending

   Sorting by 'Score' descending puts highest scores first for each ID.

2. Step 2: Drop duplicates by ID keeping first

   After sorting, dropping duplicates by 'ID' keeps the first (highest score) row per ID.

3. Final Answer:

   df.sort_values('Score', ascending=False).drop_duplicates(subset=['ID']) -> Option D

4. Quick Check:

   Sort then drop duplicates keeps highest score per ID [OK]

Hint: Sort by score descending before drop_duplicates to keep max [OK]

Common Mistakes:

• Not sorting before dropping duplicates
• Dropping duplicates on wrong column
• Using keep='last' without sorting