Handling duplicate column names in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of DataFrame with duplicate columns after selection

What is the output of the following code snippet when selecting column 'A' from a DataFrame with duplicate column names?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({"A": [1, 2], "B": [3, 4], "A": [5, 6]})
result = df["A"]
print(result)

A [5, 6]

B KeyError: 'A'

C [1, 2]

D DataFrame with two columns named 'A'

💡 Hint

When duplicate column names exist, the last one overwrites previous ones in dictionary initialization.

In pandas, when creating a DataFrame from a dictionary with duplicate keys, the last key-value pair is used. So the column 'A' contains [5, 6]. Selecting df['A'] returns that column as a Series.

❓ data_output

intermediate

Number of columns after reading CSV with duplicate headers

Given a CSV file with headers: 'X,Y,X', what will be the number of columns in the DataFrame after reading it with pandas default settings?

Data Analysis Python

import pandas as pd
from io import StringIO

csv_data = "X,Y,X\n1,2,3\n4,5,6"
df = pd.read_csv(StringIO(csv_data))
print(len(df.columns))

A Raises an error

B 2

C 1

D 3

💡 Hint

pandas allows duplicate column names by default when reading CSV files.

pandas does not automatically rename duplicate columns when reading CSV files. It keeps all columns, so the DataFrame has 3 columns.

🔧 Debug

advanced

Identify error when accessing duplicate columns by attribute

What error occurs when trying to access a duplicate column by attribute in pandas DataFrame?

Data Analysis Python

import pandas as pd

df = pd.DataFrame([[1, 2], [3, 4]], columns=["A", "A"])
print(df.A)

A AttributeError: 'DataFrame' object has no attribute 'A'

B Returns a DataFrame with both 'A' columns

C Raises a KeyError

D Returns the last 'A' column as a Series

💡 Hint

Attribute access raises AttributeError when duplicate column names exist.

In pandas, when duplicate columns exist, attribute access (df.A) raises AttributeError because it requires unique column names.

🚀 Application

advanced

Resolving duplicate columns after concatenation

After concatenating two DataFrames with overlapping column names, which method correctly renames duplicate columns to unique names?

Data Analysis Python

import pandas as pd

df1 = pd.DataFrame({"A": [1], "B": [2]})
df2 = pd.DataFrame({"A": [3], "B": [4]})
df_concat = pd.concat([df1, df2], axis=1)
# Which code renames duplicates correctly?

A df_concat.columns = list(set(df_concat.columns))

B df_concat.columns = df_concat.columns.unique()

C df_concat.columns = [f'{col}_{i}' for i, col in enumerate(df_concat.columns)]

D df_concat.columns = df_concat.columns.drop_duplicates()

💡 Hint

Use enumeration to append index to each column name for uniqueness.

Option C appends an index to each column name, making all column names unique. Other options either remove duplicates or cause errors.

🧠 Conceptual

expert

Effect of duplicate columns on groupby aggregation

If a DataFrame has duplicate column names and you perform a groupby aggregation on one of these columns, what is the expected behavior?

A Aggregation applies only to the last occurrence of the column name

B Aggregation applies to all columns with that name, returning multiple results per group

C Aggregation applies only to the first occurrence of the column name

D Raises a ValueError due to ambiguous column names

💡 Hint

pandas treats columns as labels; duplicate names mean multiple columns with same label.

When grouping by a column name that appears multiple times, pandas applies aggregation to all columns with that name, resulting in multiple aggregated columns.

Practice

1. What happens if a pandas DataFrame has duplicate column names and you try to select one column by name using df['column_name']?

easy

A. It merges the columns with the same name into one.

B. It returns all columns with that name as a DataFrame.

C. It returns only the first column with that name.

D. It raises a KeyError.

Solution

1. Step 1: Understand pandas behavior with duplicate columns

   When selecting a column by name, pandas returns all columns with that name as a DataFrame if duplicates exist.

2. Step 2: Confirm selection result type

   Instead of a Series, pandas returns a DataFrame containing all matching columns.

3. Final Answer:

   It returns all columns with that name as a DataFrame. -> Option B

4. Quick Check:

   Duplicate columns selection = DataFrame [OK]

Hint: Selecting duplicate columns returns all as DataFrame, not error [OK]

Common Mistakes:

• Assuming it raises an error
• Thinking it returns only the first column
• Believing columns merge automatically

2. Which of the following is the correct way to rename duplicate columns in a pandas DataFrame df to make them unique?

easy

A. df.columns = pd.io.parsers.ParserBase({'names':df.columns})._maybe_dedup_names(df.columns)

B. df.columns = df.columns + '_1'

C. df.rename_duplicates()

D. df.columns = list(set(df.columns))

Solution

1. Step 1: Identify pandas method for deduplication

   pandas internal method _maybe_dedup_names can rename duplicate columns to unique names.

2. Step 2: Check each option

   A uses pandas internal deduplication correctly; B adds suffix to all columns, not just duplicates; C is invalid method; D removes duplicates losing data.

3. Final Answer:

   df.columns = pd.io.parsers.ParserBase({'names':df.columns})._maybe_dedup_names(df.columns) -> Option A

4. Quick Check:

   Use pandas internal deduplication = df.columns = pd.io.parsers.ParserBase({'names':df.columns})._maybe_dedup_names(df.columns) [OK]

Hint: Use pandas internal _maybe_dedup_names to rename duplicates [OK]

Common Mistakes:

• Trying to rename with set() which loses order and data
• Using non-existent method rename_duplicates()
• Adding suffix to all columns blindly

3. Given the DataFrame df with columns ['A', 'B', 'A'] and data:

   A  B  A
0  1  2  3
1  4  5  6

What is the output of df['A']?

medium

A. A Series with values [1, 4]

B. A DataFrame with columns 'A' and 'B'

C. A DataFrame with two columns both named 'A' containing values [[1, 3], [4, 6]]

D. Raises a KeyError

Solution

1. Step 1: Check columns and duplicates

   DataFrame has duplicate columns named 'A' at positions 0 and 2.

2. Step 2: Selecting df['A'] returns all columns named 'A'

   pandas returns a DataFrame with both 'A' columns, values [[1, 3], [4, 6]].

3. Final Answer:

   A DataFrame with two columns both named 'A' containing values [[1, 3], [4, 6]] -> Option C

4. Quick Check:

   Duplicate column selection = DataFrame with both columns [OK]

Hint: Selecting duplicate columns returns DataFrame with all matching columns [OK]

Common Mistakes:

• Expecting a single Series output
• Thinking it raises an error
• Confusing columns returned with all DataFrame columns

4. You have a DataFrame df with duplicate columns. You try to rename duplicates by running:

df.columns = df.columns + '_dup'

but it renames all columns, not just duplicates. How to fix this?

medium

A. Use df.columns = pd.io.parsers.ParserBase({'names':df.columns})._maybe_dedup_names(df.columns).

B. Drop duplicate columns before renaming.

C. Use df.rename(columns=lambda x: x+'_dup').

D. Use a loop to add suffix only to duplicate columns.

Solution

1. Step 1: Understand why all columns get renamed

   Adding '_dup' suffix to all columns renames every column, not just duplicates.

2. Step 2: Fix by renaming only duplicates

   Use a loop or comprehension to detect duplicates and rename only those columns.

3. Final Answer:

   Use a loop to add suffix only to duplicate columns. -> Option D

4. Quick Check:

   Rename duplicates selectively = loop method [OK]

Hint: Rename only duplicates by looping and checking counts [OK]

Common Mistakes:

• Renaming all columns blindly
• Using rename() without condition
• Dropping duplicates losing data

5. You have a DataFrame df with columns ['X', 'Y', 'X', 'Z', 'Y']. You want to rename duplicates by adding suffixes _1, _2, ... to make all column names unique. Which code correctly achieves this?

hard

A. df.columns = pd.Series(df.columns).groupby(df.columns).cumcount().add(1).astype(str).radd(df.columns + '_')

B. df.columns = pd.Series(df.columns).groupby(df.columns).cumcount().astype(str) + df.columns

C. df.columns = [f'{col}_{i}' for i, col in enumerate(df.columns)]

D. df.columns = list(set(df.columns))

Solution

1. Step 1: Understand the goal

   We want to add suffixes _1, _2, ... to duplicate columns to make names unique.

2. Step 2: Analyze each option

   A correctly groups by column names and adds suffixes starting at 1; B adds suffix before column name incorrectly; C adds suffixes by position, not duplicates; D removes duplicates losing data.

3. Final Answer:

   df.columns = pd.Series(df.columns).groupby(df.columns).cumcount().add(1).astype(str).radd(df.columns + '_') -> Option A

4. Quick Check:

   Groupby cumcount + suffix = unique names [OK]

Hint: Use groupby cumcount to add numeric suffixes to duplicates [OK]

Common Mistakes:

• Using set() which loses order and data
• Adding suffixes by position, not duplicates
• Ignoring duplicates and renaming all columns