duplicated() for finding duplicates in Pandas - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of duplicated() with default parameters

What is the output of the following code snippet?

Pandas

import pandas as pd

df = pd.DataFrame({'A': [1, 2, 2, 3, 4, 4, 4], 'B': ['x', 'y', 'y', 'z', 'x', 'x', 'x']})
result = df.duplicated()
print(result.tolist())

A [True, False, True, False, True, False, True]

B [False, True, True, False, True, True, True]

C [False, False, False, False, False, False, False]

D [False, False, True, False, False, True, True]

💡 Hint

duplicated() marks all rows that have appeared before as True, except the first occurrence.

The duplicated() method by default marks all rows that are duplicates of a previous row as True. The first occurrence is marked False.

In the DataFrame, the second '2' in column A is a duplicate, so True. The second and third '4's are duplicates, so True.

❓ data_output

intermediate

Count of duplicated rows with subset and keep='last'

Given the DataFrame below, how many rows are marked as duplicates when using subset=['A'] and keep='last'?

Pandas

import pandas as pd

df = pd.DataFrame({'A': [5, 6, 5, 7, 6, 8, 5]})
duplicates = df.duplicated(subset=['A'], keep='last')
count = duplicates.sum()
print(count)

A 3

B 2

C 1

D 4

💡 Hint

keep='last' marks all duplicates except the last occurrence as True.

Values 5 and 6 appear multiple times. With keep='last', the first occurrences are marked True (duplicates), and the last occurrence is False.

For 5: positions 0 and 2 are duplicates (True), position 6 is last (False). For 6: position 1 is duplicate (True), position 4 is last (False). Total duplicates: 3.

🔧 Debug

advanced

Identify the error in duplicated() usage

What error will this code raise?

Pandas

import pandas as pd

df = pd.DataFrame({'A': [1, 2, 3]})
result = df.duplicated(keep='middle')
print(result)

A ValueError: keep must be either 'first', 'last', or False

B TypeError: duplicated() got an unexpected keyword argument 'keep'

C SyntaxError: invalid syntax

D No error, prints a boolean Series

💡 Hint

Check the allowed values for the keep parameter in duplicated().

The keep parameter only accepts 'first', 'last', or False. 'middle' is invalid and causes a ValueError.

🚀 Application

advanced

Filter unique rows using duplicated()

Which code snippet correctly filters the DataFrame to keep only unique rows (no duplicates) based on all columns?

Pandas

import pandas as pd

df = pd.DataFrame({'A': [1, 2, 2, 3], 'B': ['a', 'b', 'b', 'c']})

A df[df.duplicated(keep=False)]

B df[df.duplicated(keep='first')]

C df[~df.duplicated(keep=False)]

D df[~df.duplicated(keep='last')]

💡 Hint

Use keep=False to mark all duplicates as True, then invert to get unique rows.

duplicated(keep=False) marks all duplicates as True. Using ~ inverts the mask to select only unique rows.

Option C correctly filters unique rows.

🧠 Conceptual

expert

Understanding duplicated() behavior with subset and keep=False

Consider a DataFrame with columns 'X' and 'Y'. If you run df.duplicated(subset=['X'], keep=False), what does the output represent?

A Marks only the first occurrence of each duplicate value in column 'X' as True.

B Marks all rows that have duplicate values in column 'X', including the first and last occurrences.

C Marks only the last occurrence of each duplicate value in column 'X' as True.

D Marks no rows as duplicates because keep=False disables duplicate detection.

💡 Hint

keep=False means no row is considered unique among duplicates.

With keep=False, duplicated() marks all duplicates as True, including the first and last occurrences.

Using subset=['X'] limits the check to column 'X'.