Why duplicate detection matters in Pandas - Test Your Understanding

Practice - 5 Tasks

1fill in blank

easy

Complete the code to check for duplicate rows in the DataFrame.

Pandas

duplicates = df.[1]()

Drag options to blanks, or click blank then click option'

Aduplicated

Bdrop_duplicates

Cunique

Disnull

Attempts:

3 left

💡 Hint

Common Mistakes

Using drop_duplicates() removes duplicates instead of identifying them.

Using unique() is for unique values, not rows.

Using isnull() checks for missing values, not duplicates.

The duplicated() method returns a boolean Series showing which rows are duplicates.

2fill in blank

medium

Complete the code to count how many duplicate rows exist in the DataFrame.

Pandas

num_duplicates = df.duplicated().[1]()

Drag options to blanks, or click blank then click option'

Amean

Bsum

Ccount

Dsize

Attempts:

3 left

💡 Hint

Common Mistakes

Using count() counts all rows, not just duplicates.

Using mean() gives the fraction of duplicates, not the count.

Using size returns total number of rows.

Summing the boolean Series counts how many True values (duplicates) there are.

3fill in blank

hard

Fix the error in the code to remove duplicate rows from the DataFrame.

Pandas

df_clean = df.[1](keep='first')

Drag options to blanks, or click blank then click option'

Aunique

Bduplicated

Cdrop_duplicates

Ddropna

Attempts:

3 left

💡 Hint

Common Mistakes

Using duplicated() returns a boolean Series, not a DataFrame.

Using unique() is for unique values, not rows.

Using dropna() removes missing values, not duplicates.

The drop_duplicates() method removes duplicate rows, keeping the first occurrence.

4fill in blank

hard

Fill both blanks to create a dictionary of word lengths for words longer than 3 characters.

Pandas

lengths = {word: [1] for word in words if len(word) [2] 3}

Drag options to blanks, or click blank then click option'

Alen(word)

B>

C<

Dword

Attempts:

3 left

💡 Hint

Common Mistakes

Using '<' instead of '>' filters shorter words.

Using 'word' as value stores the word, not its length.

The dictionary comprehension maps each word to its length if the word length is greater than 3.

5fill in blank

hard

Fill all three blanks to create a dictionary with uppercase keys and values greater than zero.

Pandas

result = [1]: [2] for k, v in data.items() if v [3] 0}

Drag options to blanks, or click blank then click option'

Ak.upper()

Bv

C>

Dk

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'k' instead of 'k.upper()' keeps keys lowercase.

Using '<' instead of '>' filters values less than zero.

Using 'k' as value instead of 'v' stores keys as values.

This dictionary comprehension creates keys as uppercase versions of k and includes only items where the value v is greater than zero.

Practice

1. Why is detecting duplicates important in data analysis using pandas?

easy

A. Duplicates help in adding more rows to the dataset.

B. Duplicates make the data look colorful in plots.

C. Duplicates can cause incorrect results and bias in analysis.

D. Duplicates increase the speed of data processing.

Solution

1. Step 1: Understand the impact of duplicates on data

   Duplicates repeat the same information, which can skew calculations like averages or counts.

2. Step 2: Recognize the importance of accuracy

   Removing duplicates ensures the analysis reflects true data without repeated bias.

3. Final Answer:

   Duplicates can cause incorrect results and bias in analysis. -> Option C

4. Quick Check:

   Duplicates cause bias [OK]

Hint: Duplicates distort results; always check before analysis [OK]

Common Mistakes:

• Thinking duplicates speed up processing
• Believing duplicates add useful new data
• Ignoring duplicates in analysis

2. Which pandas function is used to find duplicate rows in a DataFrame?

easy

A. df.duplicate()

B. df.duplicated()

C. df.find_duplicates()

D. df.get_duplicates()

Solution

1. Step 1: Recall pandas functions for duplicates

   The correct function to detect duplicates is duplicated(), which returns a boolean Series.

2. Step 2: Verify function names

   Other options like duplicate(), find_duplicates(), and get_duplicates() do not exist in pandas.

3. Final Answer:

   df.duplicated() -> Option B

4. Quick Check:

   Correct function is duplicated() [OK]

Hint: Remember: duplicated() returns True for duplicate rows [OK]

Common Mistakes:

• Using non-existent functions like duplicate()
• Confusing duplicated() with drop_duplicates()
• Misspelling the function name

3. Given the DataFrame:

import pandas as pd
data = {'Name': ['Anna', 'Bob', 'Anna', 'Cody'], 'Age': [25, 30, 25, 22]}
df = pd.DataFrame(data)
print(df.duplicated())

What is the output?

medium

A. [True, True, False, False]

B. [True, False, True, False]

C. [False, True, False, True]

D. [False, False, True, False]

Solution

1. Step 1: Understand df.duplicated() output

   This function marks True for rows that are duplicates of a previous row.

2. Step 2: Check each row for duplication

   Row 0 (Anna,25) is first occurrence -> False.
   Row 1 (Bob,30) is unique -> False.
   Row 2 (Anna,25) repeats row 0 -> True.
   Row 3 (Cody,22) is unique -> False.

3. Final Answer:

   [False, False, True, False] -> Option D

4. Quick Check:

   Third row duplicate = True, others False [OK]

Hint: duplicated() marks True only after first occurrence [OK]

Common Mistakes:

• Marking first occurrence as True
• Confusing duplicated() with drop_duplicates() output
• Ignoring order of rows

4. You run this code but get an error:

df = pd.DataFrame({'A': [1, 2, 2], 'B': [3, 4, 4]})
duplicates = df.duplicate()

What is the error and how to fix it?

medium

A. AttributeError: 'DataFrame' object has no attribute 'duplicate'; fix by using df.duplicated()

B. SyntaxError: missing parentheses; fix by adding () after duplicate

C. TypeError: duplicate() requires arguments; fix by passing columns

D. No error; code runs fine

Solution

1. Step 1: Identify the error type

   The error is AttributeError because duplicate() is not a valid pandas method.

2. Step 2: Correct the method name

   The correct method to detect duplicates is duplicated() with a 'd' at the end.

3. Final Answer:

   AttributeError: 'DataFrame' object has no attribute 'duplicate'; fix by using df.duplicated() -> Option A

4. Quick Check:

   Use duplicated(), not duplicate() [OK]

Hint: Check method spelling: duplicated() not duplicate() [OK]

Common Mistakes:

• Using duplicate() instead of duplicated()
• Assuming no error occurs
• Trying to pass arguments to duplicate()

5. You have a DataFrame with columns ['ID', 'Name', 'Score']. Some rows have the same ID but different Score. You want to keep only the first occurrence of each ID and remove duplicates based on ID only. Which code correctly does this?

hard

A. df.drop_duplicates(subset=['ID'], keep='first')

B. df.drop_duplicates(keep='first')

C. df.duplicated(subset=['ID'], keep='first')

D. df.drop_duplicates(subset=['Score'], keep='last')

Solution

1. Step 1: Understand the goal

   We want to remove duplicates based on ID only, keeping the first occurrence.

2. Step 2: Choose correct method and parameters

   drop_duplicates() removes duplicates; subset=['ID'] limits check to ID column; keep='first' keeps first occurrence.

3. Step 3: Evaluate options

   df.drop_duplicates(subset=['ID'], keep='first') matches the requirement.
   df.drop_duplicates(keep='first') checks all columns, not just ID.
   df.duplicated(subset=['ID'], keep='first') returns boolean Series, not DataFrame.
   df.drop_duplicates(subset=['Score'], keep='last') removes duplicates by Score, not ID.

4. Final Answer:

   df.drop_duplicates(subset=['ID'], keep='first') -> Option A

5. Quick Check:

   Drop duplicates by ID, keep first [OK]

Hint: Use subset to specify columns for duplicate check [OK]

Common Mistakes:

• Not using subset to limit columns
• Using duplicated() instead of drop_duplicates() to remove rows
• Keeping last instead of first occurrence