Setting a column as index in Pandas - Time & Space Complexity

Time Complexity: Setting a column as index

O(n)

Understanding Time Complexity

When we set a column as the index in pandas, we change how data is organized internally.

We want to know how the time it takes grows as the data gets bigger.

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

import pandas as pd

n = 10  # Example value for n

df = pd.DataFrame({
    'A': range(n),
    'B': range(n, 0, -1),
    'C': ['x'] * n
})

# Set column 'A' as index
indexed_df = df.set_index('A')

This code creates a DataFrame with n rows and then sets column 'A' as the index.

Identify Repeating Operations

• Primary operation: pandas goes through each row to assign the new index values.
• How many times: Once for each of the n rows in the DataFrame.

How Execution Grows With Input

As the number of rows increases, the work to set the index grows in a straight line.

Input Size (n)	Approx. Operations
10	About 10 operations
100	About 100 operations
1000	About 1000 operations

Pattern observation: Doubling the rows roughly doubles the work needed.

Final Time Complexity

Time Complexity: O(n)

This means the time to set the index grows directly with the number of rows.

Common Mistake

[X] Wrong: "Setting a column as index is instant no matter how big the data is."

[OK] Correct: pandas must look at each row to assign the index, so bigger data takes more time.

Interview Connect

Understanding how data operations scale helps you explain your code choices clearly and confidently.

Self-Check

"What if we set multiple columns as a multi-index? How would the time complexity change?"

Practice

1. What does the set_index() method do in pandas?

easy

A. It sorts the DataFrame by a column.

B. It deletes a column from the DataFrame.

C. It adds a new column to the DataFrame.

D. It sets a specified column as the index of the DataFrame.

Solution

1. Step 1: Understand the purpose of set_index()

   The set_index() method changes the row labels to the values of a specified column.

2. Step 2: Compare with other options

   Options A, B, and D describe different DataFrame operations, not setting the index.

3. Final Answer:

   It sets a specified column as the index of the DataFrame. -> Option D

4. Quick Check:

   set_index() changes row labels to a column [OK]

Hint: Remember: set_index() changes row labels to a column [OK]

Common Mistakes:

• Confusing set_index() with drop() method
• Thinking set_index() adds a new column
• Assuming set_index() sorts data

2. Which of the following is the correct syntax to set the column 'Name' as the index of DataFrame df without modifying df itself?

easy

A. df.set_index('Name', inplace=True)

B. df.set_index('Name')

C. df.set_index(Name, inplace=False)

D. df.set_index('Name', drop=False)

Solution

1. Step 1: Understand inplace parameter

   By default, inplace=False, so df.set_index('Name') returns a new DataFrame without changing df.

2. Step 2: Analyze options

   df.set_index('Name', inplace=True) modifies df inplace. df.set_index('Name', inplace=False) is invalid because inplace=False is default and usually omitted. df.set_index('Name', drop=False) keeps the column but does not specify inplace, so it returns a new DataFrame.

3. Final Answer:

   df.set_index('Name') -> Option B

4. Quick Check:

   Default inplace=False means returns new DataFrame [OK]

Hint: Omit inplace to keep original DataFrame unchanged [OK]

Common Mistakes:

• Using inplace=True when wanting a new DataFrame
• Confusing drop parameter with inplace
• Adding invalid parameters

3. Given the DataFrame df below:

   ID  Score
0  101     88
1  102     92
2  103     85

What will be the output of df.set_index('ID')?

medium

A. ID Score
0 101 88
1 102 92
2 103 85

B. ID Score
0 101 88
1 102 92
2 103 85

C. Score
ID
101 88
102 92
103 85

D. Score
ID
0 88
1 92
2 85

Solution

1. Step 1: Understand set_index('ID')

   This sets the 'ID' column as the index, so 'ID' values become row labels and are no longer columns.

2. Step 2: Check output format

   The resulting DataFrame shows 'Score' as the only column, indexed by 'ID' values 101, 102, 103.

3. Final Answer:

   Score ID 101 88 102 92 103 85 -> Option C

4. Quick Check:

   set_index moves column to index =

        Score
   ID          
   101      88
   102      92
   103      85

   [OK]

Hint: Index column moves from data to row labels [OK]

Common Mistakes:

• Expecting the index column to remain as a data column
• Confusing output with original DataFrame
• Misreading index as a new column

4. You run the code:

df.set_index('Age', inplace=True)

but get a KeyError: 'Age'. What is the most likely cause?

medium

A. The column 'Age' does not exist in the DataFrame.

B. You forgot to import pandas.

C. You need to reset the index before setting a new one.

D. The inplace parameter must be False.

Solution

1. Step 1: Understand KeyError meaning

   A KeyError means the specified key (column) is not found in the DataFrame.

2. Step 2: Analyze options

   The column 'Age' does not exist in the DataFrame. correctly identifies the missing column. You forgot to import pandas. is unrelated to KeyError. You need to reset the index before setting a new one. is unnecessary; you can set any existing column as index. The inplace parameter must be False. is incorrect; inplace can be True or False.

3. Final Answer:

   The column 'Age' does not exist in the DataFrame. -> Option A

4. Quick Check:

   KeyError means missing column [OK]

Hint: Check column names before set_index to avoid KeyError [OK]

Common Mistakes:

• Assuming inplace=True causes KeyError
• Ignoring column name typos
• Thinking reset_index is required first

5. You have a DataFrame df with columns ['City', 'Population', 'Area']. You want to set 'City' as the index but keep it as a column too. Which code achieves this?

hard

A. df.set_index('City', drop=False, inplace=True)

B. df.set_index('City', inplace=True)

C. df.set_index('City', drop=True, inplace=False)

D. df.set_index('City', keep=True)

Solution

1. Step 1: Understand drop parameter

   By default, drop=True removes the column from data when setting it as index. To keep it, use drop=False.

2. Step 2: Apply inplace to modify original DataFrame

   Using inplace=True changes df directly.

3. Step 3: Analyze options

   df.set_index('City', drop=False, inplace=True) correctly uses drop=False and inplace=True. df.set_index('City', inplace=True) drops the column. df.set_index('City', drop=True, inplace=False) returns new DataFrame but drops column. df.set_index('City', keep=True) uses invalid parameter keep.

4. Final Answer:

   df.set_index('City', drop=False, inplace=True) -> Option A

5. Quick Check:

   drop=False keeps column when setting index [OK]

Hint: Use drop=False to keep column when setting index [OK]

Common Mistakes:

• Forgetting drop=False to keep the column
• Using invalid parameters like keep=True
• Not using inplace when wanting to modify original DataFrame