Why indexing matters in Pandas - Challenge Your Understanding

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of selecting rows with index labels

What is the output of this code snippet?

import pandas as pd

df = pd.DataFrame({
    'A': [10, 20, 30],
    'B': [100, 200, 300]
}, index=['x', 'y', 'z'])

result = df.loc[['y', 'z'], 'A']
print(result)

Pandas

import pandas as pd

df = pd.DataFrame({
    'A': [10, 20, 30],
    'B': [100, 200, 300]
}, index=['x', 'y', 'z'])

result = df.loc[['y', 'z'], 'A']
print(result)

A KeyError: "['y', 'z'] not in index"

B

0    20
1    30
Name: A, dtype: int64

C

x    10
y    20
Name: A, dtype: int64

D

y    20
z    30
Name: A, dtype: int64

💡 Hint

Remember that loc uses the index labels, not integer positions.

The loc method selects rows by their index labels. Since the DataFrame has index labels 'x', 'y', 'z', selecting ['y', 'z'] returns the rows with those labels. The column 'A' values for 'y' and 'z' are 20 and 30 respectively.

❓ data_output

intermediate

Number of rows after resetting index

Given this DataFrame, what is the number of rows after resetting the index?

import pandas as pd

df = pd.DataFrame({
    'score': [88, 92, 95],
    'grade': ['B', 'A', 'A']
}, index=['s1', 's2', 's3'])

new_df = df.reset_index()
print(len(new_df))

Pandas

import pandas as pd

df = pd.DataFrame({
    'score': [88, 92, 95],
    'grade': ['B', 'A', 'A']
}, index=['s1', 's2', 's3'])

new_df = df.reset_index()
print(len(new_df))

A 4

B 3

C 2

D KeyError

💡 Hint

Resetting the index adds the old index as a column but does not change the number of rows.

The reset_index() method moves the index into a column and creates a new default integer index. The number of rows remains the same, which is 3.

🔧 Debug

advanced

Why does this code raise a KeyError?

Consider this code:

import pandas as pd

df = pd.DataFrame({
    'value': [5, 10, 15]
}, index=[1, 2, 3])

print(df.loc[0])

Why does it raise a KeyError?

Pandas

import pandas as pd

df = pd.DataFrame({
    'value': [5, 10, 15]
}, index=[1, 2, 3])

print(df.loc[0])

A Because the DataFrame has no columns named 0

B Because loc only accepts column names, not index labels

C Because 0 is not in the DataFrame's index labels

D Because loc requires integer positions, not labels

💡 Hint

Check the index labels of the DataFrame and what loc expects.

The DataFrame's index labels are [1, 2, 3]. Using loc[0] tries to access the row with label 0, which does not exist, causing a KeyError.

🚀 Application

advanced

Selecting rows efficiently with index

You have a large DataFrame with a unique index of user IDs. Which method is the fastest to select multiple users by their IDs?

A Use <code>df.loc[list_of_user_ids]</code> to select rows by index labels

B Use <code>df.iloc[list_of_user_ids]</code> to select rows by integer position

C Use <code>df[df['user_id'].isin(list_of_user_ids)]</code> to filter rows

D Use <code>df.query('user_id in @list_of_user_ids')</code> to filter rows

💡 Hint

Think about how pandas uses the index for fast lookups.

Using df.loc[list_of_user_ids] leverages the index for fast direct access. The other methods involve filtering or positional indexing, which are slower for large data.

🧠 Conceptual

expert

Why setting an index improves join performance

Why does setting a column as the index improve the performance of joining two DataFrames on that column?

A Because indexes are implemented as hash tables or trees, allowing faster lookups during joins

B Because setting an index compresses the data, reducing memory usage during joins

C Because setting an index sorts the data, and joins require sorted data

D Because joins only work on index columns, so setting the index is mandatory

💡 Hint

Think about how indexes help find matching rows quickly.

Indexes in pandas are often implemented using hash tables or balanced trees, which allow quick lookup of matching keys during joins. This speeds up the join operation compared to scanning all rows.

Practice

1. Why is indexing important in pandas DataFrames?

easy

A. It deletes duplicate rows.

B. It changes the data types of columns.

C. It sorts the DataFrame automatically.

D. It labels rows to find data quickly.

Solution

1. Step 1: Understand the role of indexing

   Indexing assigns labels to rows, making it easier to locate and access data quickly.

2. Step 2: Compare with other options

   Changing data types, sorting, or deleting duplicates are different operations unrelated to indexing.

3. Final Answer:

   It labels rows to find data quickly. -> Option D

4. Quick Check:

   Indexing = labeling rows [OK]

Hint: Indexing helps find rows fast by labeling them [OK]

Common Mistakes:

• Confusing indexing with sorting
• Thinking indexing changes data types
• Assuming indexing deletes duplicates

2. Which of the following is the correct way to set the 'Name' column as the index of a DataFrame df?

easy

A. df.setIndex('Name')

B. df.index('Name')

C. df.set_index('Name')

D. df.index = 'Name'

Solution

1. Step 1: Recall the correct pandas method

   The method to set a column as index is set_index, called as df.set_index('column_name').

2. Step 2: Check other options for syntax errors

   df.index('Name') and df.setIndex('Name') are invalid methods. Assigning df.index = 'Name' sets index to a string, not a column.

3. Final Answer:

   df.set_index('Name') -> Option C

4. Quick Check:

   Use set_index('column') to set index [OK]

Hint: Use df.set_index('column') to set index [OK]

Common Mistakes:

• Using wrong method names like setIndex
• Trying to call df.index as a function
• Assigning a string directly to df.index

3. Given the DataFrame df below:

   Name  Age  Score
0  Alice   25     88
1    Bob   30     92
2  Carol   22     85

What is the output of df.loc[1, 'Score']?

medium

A. 30

B. 92

C. Bob

D. 85

Solution

1. Step 1: Understand loc usage

   loc selects data by label. Here, row label 1 and column 'Score' is requested.

2. Step 2: Find value at row 1, column 'Score'

   Row 1 corresponds to Bob, and his Score is 92.

3. Final Answer:

   92 -> Option B

4. Quick Check:

   df.loc[1, 'Score'] = 92 [OK]

Hint: loc uses row labels, so df.loc[1, 'Score'] = 92 [OK]

Common Mistakes:

• Confusing loc with iloc (position vs label)
• Selecting wrong column value
• Assuming row 1 is index 0

4. What is wrong with this code snippet?

df = pd.DataFrame({'A': [1,2], 'B': [3,4]})
df.set_index('C')

medium

A. Column 'C' does not exist, causing an error.

B. set_index cannot be used on DataFrames.

C. The code will run but not change the index.

D. The DataFrame is empty, so set_index fails.

Solution

1. Step 1: Check if column 'C' exists

   The DataFrame has columns 'A' and 'B' only; 'C' is missing.

2. Step 2: Understand set_index behavior

   Trying to set index to a non-existent column raises a KeyError.

3. Final Answer:

   Column 'C' does not exist, causing an error. -> Option A

4. Quick Check:

   Missing column in set_index causes error [OK]

Hint: Check if column exists before set_index [OK]

Common Mistakes:

• Assuming set_index works without error on missing columns
• Thinking set_index can't be used on DataFrames
• Ignoring error messages

5. You have a DataFrame df with columns 'City', 'Year', and 'Population'. You want to quickly select all rows for the city 'Paris' using indexing. Which steps correctly prepare and select the data?

hard

A. df.set_index('City', inplace=True)\ndf.loc['Paris']

B. df.set_index('Year', inplace=True)\ndf.loc['Paris']

C. df.set_index('Population', inplace=True)\ndf.loc['Paris']

D. df.loc[df['City'] == 'Paris']

Solution

1. Step 1: Set 'City' as the index

   Setting 'City' as index allows direct row selection by city name.

2. Step 2: Use loc to select rows labeled 'Paris'

   With 'City' as index, df.loc['Paris'] returns all rows for Paris efficiently.

3. Final Answer:

   df.set_index('City', inplace=True)\ndf.loc['Paris'] -> Option A

4. Quick Check:

   Index by City, then loc['Paris'] selects rows [OK]

Hint: Set index to City, then use loc['Paris'] to select [OK]

Common Mistakes:

• Setting wrong column as index
• Using loc with non-index column
• Not setting index before loc selection