MultiIndex (hierarchical indexing) in Data Analysis Python

Introduction

MultiIndex lets you organize data with multiple levels of labels. It helps you work with complex tables easily.

You have data grouped by categories and subcategories, like sales by country and city.

You want to analyze data at different levels, such as yearly and monthly data together.

You need to store and access data with multiple keys, like product type and color.

You want to perform operations like grouping or slicing on multiple index levels.

Syntax

Data Analysis Python

pd.MultiIndex.from_tuples(tuples, names=None)
pd.MultiIndex.from_product(iterables, names=None)

# Example:
index = pd.MultiIndex.from_tuples([('A', 1), ('A', 2), ('B', 1)], names=['letter', 'number'])

Use from_tuples when you have a list of label pairs or triples.

Use from_product to create all combinations from multiple lists.

Examples

This creates a MultiIndex with two levels: Country and State.

Data Analysis Python

import pandas as pd

# Create MultiIndex from tuples
index = pd.MultiIndex.from_tuples(
    [('USA', 'NY'), ('USA', 'CA'), ('Canada', 'ON')],
    names=['Country', 'State']
)
print(index)

This creates all combinations of letters and numbers as a MultiIndex.

Data Analysis Python

import pandas as pd

# Create MultiIndex from product
index = pd.MultiIndex.from_product(
    [['A', 'B'], [1, 2]],
    names=['Letter', 'Number']
)
print(index)

DataFrame with MultiIndex rows showing categories and items.

Data Analysis Python

import pandas as pd

# Use MultiIndex in DataFrame
index = pd.MultiIndex.from_tuples(
    [('Fruit', 'Apple'), ('Fruit', 'Banana'), ('Veg', 'Carrot')],
    names=['Category', 'Item']
)
data = [10, 20, 30]
df = pd.DataFrame(data, index=index, columns=['Quantity'])
print(df)

Sample Program

This program creates a DataFrame with a MultiIndex for country and state. It then shows how to access data by country and by specific state.

Data Analysis Python

import pandas as pd

# Create MultiIndex from tuples
index = pd.MultiIndex.from_tuples(
    [('USA', 'NY'), ('USA', 'CA'), ('Canada', 'ON')],
    names=['Country', 'State']
)

# Create DataFrame with MultiIndex
data = [100, 200, 300]
df = pd.DataFrame(data, index=index, columns=['Sales'])

print("DataFrame with MultiIndex:")
print(df)

# Access data for USA
print("\nSales in USA:")
print(df.loc['USA'])

# Access data for USA, NY
print("\nSales in USA, NY:")
print(df.loc[('USA', 'NY')])

Important Notes

MultiIndex makes it easy to work with grouped or hierarchical data.

You can select data by one or more levels using loc.

MultiIndex can be used for rows or columns in a DataFrame.

Summary

MultiIndex lets you label data with multiple levels.

It helps organize and access complex data easily.

You create MultiIndex using from_tuples or from_product.

Practice

1. What is the main purpose of a MultiIndex in pandas?

easy

A. To create plots automatically

B. To label data with multiple levels for better organization

C. To convert data into a list

D. To speed up numerical calculations

Solution

1. Step 1: Understand MultiIndex concept

   MultiIndex allows multiple levels of labels on rows or columns, organizing complex data.

2. Step 2: Compare options

   Only To label data with multiple levels for better organization correctly describes MultiIndex's purpose; others describe unrelated features.

3. Final Answer:

   To label data with multiple levels for better organization -> Option B

4. Quick Check:

   MultiIndex = multiple levels labeling [OK]

Hint: MultiIndex means multiple levels of labels [OK]

Common Mistakes:

• Thinking MultiIndex speeds up calculations
• Confusing MultiIndex with plotting tools
• Believing MultiIndex converts data to lists

2. Which of the following is the correct way to create a MultiIndex from tuples in pandas?

easy

A. pd.MultiIndex.build_tuples([('A', 1), ('B', 2)])

B. pd.MultiIndex.create_tuples([('A', 1), ('B', 2)])

C. pd.MultiIndex.make_tuples([('A', 1), ('B', 2)])

D. pd.MultiIndex.from_tuples([('A', 1), ('B', 2)])

Solution

1. Step 1: Recall pandas MultiIndex creation methods

   The correct method to create MultiIndex from tuples is from_tuples.

2. Step 2: Check method names

   Only pd.MultiIndex.from_tuples([('A', 1), ('B', 2)]) uses the correct method name; others are invalid pandas methods.

3. Final Answer:

   pd.MultiIndex.from_tuples([('A', 1), ('B', 2)]) -> Option D

4. Quick Check:

   Use from_tuples to create MultiIndex from tuples [OK]

Hint: Use from_tuples to create MultiIndex from tuple list [OK]

Common Mistakes:

• Using non-existent methods like create_tuples
• Confusing method names with make_tuples or build_tuples
• Forgetting to import pandas as pd

3. Given the code:

import pandas as pd
index = pd.MultiIndex.from_tuples([('A', 1), ('A', 2), ('B', 1)])
data = pd.Series([10, 20, 30], index=index)
print(data.loc['A'])

What is the output?

medium

A. A 10
A 20
dtype: int64

B. 10
20

C. 1 10
2 20
dtype: int64

D. KeyError

Solution

1. Step 1: Understand MultiIndex and selection

   The MultiIndex has two levels: first level 'A' or 'B', second level numbers. Selecting data.loc['A'] returns all entries where first level is 'A'.

2. Step 2: Check values for 'A'

   Entries with first level 'A' are ('A', 1):10 and ('A', 2):20, so output shows index 1 and 2 with values 10 and 20.

3. Final Answer:

   1 10 2 20 dtype: int64 -> Option C

4. Quick Check:

   data.loc['A'] returns second level index and values [OK]

Hint: Selecting first level returns second level index and values [OK]

Common Mistakes:

• Expecting full tuple index in output
• Getting KeyError by wrong indexing
• Confusing index labels with values

4. What is wrong with this code snippet?

import pandas as pd
index = pd.MultiIndex.from_product([['X', 'Y'], [1, 2]])
data = pd.Series([5, 10, 15], index=index)
print(data)

medium

A. The length of data does not match the MultiIndex length

B. from_product is not a valid MultiIndex method

C. Series cannot have MultiIndex

D. The index should be a list, not MultiIndex

Solution

1. Step 1: Check MultiIndex length

   from_product creates all combinations: 2 (X,Y) * 2 (1,2) = 4 index entries.

2. Step 2: Check data length

   Data has length 3, which does not match 4 index entries, causing a ValueError.

3. Final Answer:

   The length of data does not match the MultiIndex length -> Option A

4. Quick Check:

   Data length must match MultiIndex length [OK]

Hint: Data length must equal MultiIndex length [OK]

Common Mistakes:

• Thinking from_product is invalid
• Believing Series can't have MultiIndex
• Assuming index must be list, not MultiIndex

5. You have sales data for two stores over two years stored as a MultiIndex DataFrame with levels ['Store', 'Year']. How do you select sales data for Store 'B' in year 2023?

hard

A. df.loc[('B', 2023)]

B. df['B'][2023]

C. df.loc[2023, 'B']

D. df.loc['B', [2023]]

Solution

1. Step 1: Understand MultiIndex selection syntax

   To select a row with multiple index levels, use a tuple inside loc with all levels specified.

2. Step 2: Analyze options

   df.loc[('B', 2023)] correctly specifies both levels as a tuple. df.loc[2023, 'B'] uses incorrect order of levels (year before store). df['B'][2023] attempts column access first, causing KeyError. df.loc['B', [2023]] uses a list which pandas interprets as column slice, causing KeyError if no such columns.

3. Final Answer:

   df.loc[('B', 2023)] -> Option A

4. Quick Check:

   Use tuple in loc for MultiIndex row selection [OK]

Hint: Use tuple inside loc to select MultiIndex row [OK]

Common Mistakes:

• Reversing the order of index levels
• Using df['B'] to access rows instead of loc with tuple
• Passing list to loc causing column slice misinterpretation