duplicated() for finding duplicates in Pandas - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the duplicated() function in pandas do?

It returns a Boolean Series indicating whether each row is a duplicate of a previous row in the DataFrame.

beginner

How can you use duplicated() to find all duplicate rows except the first occurrence?

By default, duplicated() marks duplicates as True except for the first occurrence, which is False.

intermediate

What parameter of duplicated() controls which duplicates are marked as True?

The keep parameter controls this. It can be 'first' (default), 'last', or False to mark all duplicates as True.

intermediate

How do you find duplicates based on specific columns using duplicated()?

Use the subset parameter to specify columns to check for duplicates instead of the entire row.

intermediate

What is the difference between duplicated() and drop_duplicates()?

duplicated() returns a Boolean mask showing duplicates, while drop_duplicates() returns a DataFrame with duplicates removed.

What does df.duplicated() return?

A A DataFrame with duplicates removed

B The count of duplicate rows

C A Boolean Series marking duplicate rows except the first occurrence

D The unique rows only

duplicated() returns a Boolean Series marking duplicates as True except the first occurrence.

Which keep parameter value marks all duplicates as True in duplicated()?

A 'last'

B False

C 'first'

D None

Setting keep=False marks all duplicates as True.

How do you check duplicates based on only some columns?

A Use <code>unique()</code> on those columns

B Use <code>drop_duplicates()</code> without parameters

C Use <code>groupby()</code> only

D Use the <code>subset</code> parameter in <code>duplicated()</code>

The subset parameter lets you specify columns to check for duplicates.

What does df.duplicated(keep='last') do?

A Marks duplicates as True except the last occurrence

B Marks duplicates as True except the first occurrence

C Marks all duplicates as True

D Removes duplicates from DataFrame

With keep='last', all duplicates except the last occurrence are marked True.

Which function returns a DataFrame with duplicates removed?

A <code>drop_duplicates()</code>

B <code>duplicated()</code>

C <code>unique()</code>

D <code>isnull()</code>

drop_duplicates() returns a DataFrame with duplicates removed.

Explain how to use duplicated() to find duplicate rows in a DataFrame and how to customize which duplicates are marked.

Think about which rows are marked True and how to change that.

Describe the difference between duplicated() and drop_duplicates() and when you might use each.

Consider output type and purpose.