MultiIndex (hierarchical indexing) in Data Analysis Python - Step-by-Step Execution

Concept Flow - MultiIndex (hierarchical indexing)

Create MultiIndex from tuples

↓

Assign MultiIndex to DataFrame

↓

Access data by outer level

↓

Access data by inner level

↓

Slice or filter using MultiIndex

We build a MultiIndex from tuples, assign it to a DataFrame, then access or slice data by levels.

Execution Sample

Data Analysis Python

import pandas as pd
index = pd.MultiIndex.from_tuples([('A', 1), ('A', 2), ('B', 1)])
df = pd.DataFrame({'Value': [10, 20, 30]}, index=index)
print(df.loc['A'])

Create a DataFrame with a MultiIndex and select rows where the first level is 'A'.

Execution Table

Step	Action	Variable/Expression	Result/Value
1	Create MultiIndex from tuples	pd.MultiIndex.from_tuples([('A',1),('A',2),('B',1)])	MultiIndex([('A', 1), ('A', 2), ('B', 1)], names=[None, None])
2	Create DataFrame with MultiIndex	pd.DataFrame({'Value':[10,20,30]}, index=index)	DataFrame with MultiIndex: Value A 1 10 2 20 B 1 30
3	Access rows with first level 'A'	df.loc['A']	DataFrame: Value 1 10 2 20
4	Access rows with second level 1 under 'A'	df.loc[('A',1)]	Series: Value 10 Name: (A, 1), dtype: int64
5	Slice rows where second level >= 2	df.loc[pd.IndexSlice[:, 2:], :]	DataFrame: Value A 2 20
6	Exit	No more operations	End of example

💡 All steps executed to show MultiIndex creation, assignment, and access.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	Final
index	None	MultiIndex([('A', 1), ('A', 2), ('B', 1)])	MultiIndex([('A', 1), ('A', 2), ('B', 1)])	Same	Same	Same	Same
df	None	None	DataFrame with MultiIndex and values	Same	Same	Same	Same
df.loc['A']	None	None	None	DataFrame with rows for 'A'	Same	Same	Same
df.loc[('A',1)]	None	None	None	None	Series with value 10	Same	Same
df.loc[pd.IndexSlice[:, 2:], :]	None	None	None	None	None	DataFrame with rows where second level >= 2	Same

Key Moments - 3 Insights

Why does df.loc['A'] return multiple rows?

Because 'A' is the first level of the MultiIndex and selecting it returns all rows where the first level is 'A', as shown in execution_table step 3.

What happens if you try df.loc['C'] which is not in the index?

It raises a KeyError because 'C' is not present in the first level of the MultiIndex, similar to how normal indexing works but with hierarchical levels.

How does slicing with pd.IndexSlice[:, 2:] work?

It selects all rows where the second level of the MultiIndex is 2 or greater, as shown in execution_table step 5.

Visual Quiz - 3 Questions

Look at the execution_table step 3, what rows does df.loc['A'] return?

A All rows in the DataFrame

B Rows where second level is 1

C Rows where first level is 'A' (two rows)

D Only the row ('A', 1)

💡 Hint

Check the 'Result/Value' column in step 3 of execution_table.

At which step does df.loc[('A',1)] return a Series?

A Step 2

B Step 4

C Step 5

D Step 3

💡 Hint

Look at the 'Action' and 'Result/Value' columns in execution_table for step 4.

If we change the tuple ('B', 1) to ('B', 3), which step's output changes?

A Step 5 output changes

B Step 3 output changes

C Step 4 output changes

D No output changes

💡 Hint

Step 5 filters rows where second level >= 2, so changing ('B',1) to ('B',3) affects this.

Concept Snapshot

MultiIndex (hierarchical indexing) lets you use multiple levels of row labels.
Create with pd.MultiIndex.from_tuples() and assign as DataFrame index.
Access data by levels using df.loc[level1] or df.loc[(level1, level2)].
Use pd.IndexSlice for slicing across levels.
It helps organize complex data with multiple keys.

Full Transcript

This example shows how to create a MultiIndex from tuples and assign it to a DataFrame. We then access data by selecting the first level 'A', which returns multiple rows. Selecting a full tuple like ('A',1) returns a single row as a Series. We also slice rows where the second level is greater or equal to 2 using pd.IndexSlice. The variable tracker shows how variables change after each step. Key moments clarify common confusions about selecting levels and slicing. The visual quiz tests understanding of these steps. MultiIndex helps organize data with multiple keys for easier access and slicing.

Practice

1. What is the main purpose of a MultiIndex in pandas?

easy

A. To create plots automatically

B. To label data with multiple levels for better organization

C. To convert data into a list

D. To speed up numerical calculations

Solution

1. Step 1: Understand MultiIndex concept

   MultiIndex allows multiple levels of labels on rows or columns, organizing complex data.

2. Step 2: Compare options

   Only To label data with multiple levels for better organization correctly describes MultiIndex's purpose; others describe unrelated features.

3. Final Answer:

   To label data with multiple levels for better organization -> Option B

4. Quick Check:

   MultiIndex = multiple levels labeling [OK]

Hint: MultiIndex means multiple levels of labels [OK]

Common Mistakes:

• Thinking MultiIndex speeds up calculations
• Confusing MultiIndex with plotting tools
• Believing MultiIndex converts data to lists

2. Which of the following is the correct way to create a MultiIndex from tuples in pandas?

easy

A. pd.MultiIndex.build_tuples([('A', 1), ('B', 2)])

B. pd.MultiIndex.create_tuples([('A', 1), ('B', 2)])

C. pd.MultiIndex.make_tuples([('A', 1), ('B', 2)])

D. pd.MultiIndex.from_tuples([('A', 1), ('B', 2)])

Solution

1. Step 1: Recall pandas MultiIndex creation methods

   The correct method to create MultiIndex from tuples is from_tuples.

2. Step 2: Check method names

   Only pd.MultiIndex.from_tuples([('A', 1), ('B', 2)]) uses the correct method name; others are invalid pandas methods.

3. Final Answer:

   pd.MultiIndex.from_tuples([('A', 1), ('B', 2)]) -> Option D

4. Quick Check:

   Use from_tuples to create MultiIndex from tuples [OK]

Hint: Use from_tuples to create MultiIndex from tuple list [OK]

Common Mistakes:

• Using non-existent methods like create_tuples
• Confusing method names with make_tuples or build_tuples
• Forgetting to import pandas as pd

3. Given the code:

import pandas as pd
index = pd.MultiIndex.from_tuples([('A', 1), ('A', 2), ('B', 1)])
data = pd.Series([10, 20, 30], index=index)
print(data.loc['A'])

What is the output?

medium

A. A 10
A 20
dtype: int64

B. 10
20

C. 1 10
2 20
dtype: int64

D. KeyError

Solution

1. Step 1: Understand MultiIndex and selection

   The MultiIndex has two levels: first level 'A' or 'B', second level numbers. Selecting data.loc['A'] returns all entries where first level is 'A'.

2. Step 2: Check values for 'A'

   Entries with first level 'A' are ('A', 1):10 and ('A', 2):20, so output shows index 1 and 2 with values 10 and 20.

3. Final Answer:

   1 10 2 20 dtype: int64 -> Option C

4. Quick Check:

   data.loc['A'] returns second level index and values [OK]

Hint: Selecting first level returns second level index and values [OK]

Common Mistakes:

• Expecting full tuple index in output
• Getting KeyError by wrong indexing
• Confusing index labels with values

4. What is wrong with this code snippet?

import pandas as pd
index = pd.MultiIndex.from_product([['X', 'Y'], [1, 2]])
data = pd.Series([5, 10, 15], index=index)
print(data)

medium

A. The length of data does not match the MultiIndex length

B. from_product is not a valid MultiIndex method

C. Series cannot have MultiIndex

D. The index should be a list, not MultiIndex

Solution

1. Step 1: Check MultiIndex length

   from_product creates all combinations: 2 (X,Y) * 2 (1,2) = 4 index entries.

2. Step 2: Check data length

   Data has length 3, which does not match 4 index entries, causing a ValueError.

3. Final Answer:

   The length of data does not match the MultiIndex length -> Option A

4. Quick Check:

   Data length must match MultiIndex length [OK]

Hint: Data length must equal MultiIndex length [OK]

Common Mistakes:

• Thinking from_product is invalid
• Believing Series can't have MultiIndex
• Assuming index must be list, not MultiIndex

5. You have sales data for two stores over two years stored as a MultiIndex DataFrame with levels ['Store', 'Year']. How do you select sales data for Store 'B' in year 2023?

hard

A. df.loc[('B', 2023)]

B. df['B'][2023]

C. df.loc[2023, 'B']

D. df.loc['B', [2023]]

Solution

1. Step 1: Understand MultiIndex selection syntax

   To select a row with multiple index levels, use a tuple inside loc with all levels specified.

2. Step 2: Analyze options

   df.loc[('B', 2023)] correctly specifies both levels as a tuple. df.loc[2023, 'B'] uses incorrect order of levels (year before store). df['B'][2023] attempts column access first, causing KeyError. df.loc['B', [2023]] uses a list which pandas interprets as column slice, causing KeyError if no such columns.

3. Final Answer:

   df.loc[('B', 2023)] -> Option A

4. Quick Check:

   Use tuple in loc for MultiIndex row selection [OK]

Hint: Use tuple inside loc to select MultiIndex row [OK]

Common Mistakes:

• Reversing the order of index levels
• Using df['B'] to access rows instead of loc with tuple
• Passing list to loc causing column slice misinterpretation