Stack and unstack in Data Analysis Python - Step-by-Step Execution

Concept Flow - Stack and unstack

Start with DataFrame

↓

Apply stack()

↓

DataFrame becomes Series with MultiIndex

↓

Apply unstack()

↓

Series with MultiIndex becomes DataFrame

↓

End

Stack turns columns into rows creating a Series with MultiIndex. Unstack reverses this, turning rows back into columns.

Execution Sample

Data Analysis Python

import pandas as pd

df = pd.DataFrame({
    'A': [1, 2],
    'B': [3, 4]
})
stacked = df.stack()
unstacked = stacked.unstack()

This code creates a simple DataFrame, stacks it to convert columns into rows, then unstacks it back to original form.

Execution Table

Step	Action	Data Structure Type	Shape/Index	Sample Data
1	Create DataFrame	DataFrame	(2 rows, 2 columns)	{(0, 'A'):1, (0, 'B'):3, (1, 'A'):2, (1, 'B'):4}
2	Apply stack()	Series with MultiIndex	(4 elements)	{(0, 'A'):1, (0, 'B'):3, (1, 'A'):2, (1, 'B'):4}
3	Apply unstack()	DataFrame	(2 rows, 2 columns)	{(0, 'A'):1, (0, 'B'):3, (1, 'A'):2, (1, 'B'):4}
4	End	DataFrame	(2 rows, 2 columns)	Same as Step 1

💡 Unstack reverses stack, restoring original DataFrame shape and data.

Variable Tracker

Variable	Start	After stack()	After unstack()
df	DataFrame with 2 rows, 2 columns	Unchanged	Unchanged
stacked	Not defined	Series with MultiIndex, 4 elements	Unchanged
unstacked	Not defined	Not defined	DataFrame same as df

Key Moments - 2 Insights

Why does stack() result in a Series with MultiIndex instead of a DataFrame?

Stack compresses columns into a single index level, so the result is a Series indexed by the original row index plus the stacked column labels, forming a MultiIndex (see execution_table Step 2).

Does unstack() always restore the original DataFrame exactly?

Unstack reverses stack only if the MultiIndex matches the original structure. If the Series has missing or extra index levels, unstack may produce a different shape (see execution_table Step 3).

Visual Quiz - 3 Questions

Look at the execution_table, what is the data structure type after applying stack()?

A DataFrame

B Series with MultiIndex

C List

D Dictionary

💡 Hint

Check Step 2 in execution_table under 'Data Structure Type'

At which step does the data shape change from (2 rows, 2 columns) to a Series with 4 elements?

A Step 2

B Step 1

C Step 3

D Step 4

💡 Hint

Look at the 'Shape/Index' column in execution_table for Step 2

If the stacked Series had missing values, how would unstack() affect the final DataFrame?

A It would produce an error

B It would remove rows with missing values

C It would fill missing values with NaN in the DataFrame

D It would ignore missing values and produce the same DataFrame

💡 Hint

Unstack creates a DataFrame with NaN where data is missing, see pandas behavior for unstack

Concept Snapshot

Stack and unstack:
- stack() converts columns into rows, creating a Series with MultiIndex.
- unstack() reverses stack, turning MultiIndex Series back into a DataFrame.
- Useful for reshaping data between wide and long formats.
- stack() reduces columns, unstack() expands them.
- Missing data after stack leads to NaN after unstack.

Full Transcript

Stack and unstack are pandas methods to reshape data. Starting with a DataFrame, stack() compresses columns into rows, resulting in a Series with a MultiIndex combining original row and column labels. This changes the shape from a 2D table to a 1D Series with hierarchical indexing. Unstack() reverses this process, taking a MultiIndex Series and expanding it back into a DataFrame with columns restored. The execution trace shows creating a DataFrame with two columns and two rows, stacking it to get a Series with four elements indexed by row and column labels, then unstacking back to the original DataFrame shape. Key points include understanding that stack changes the data structure type and shape, and unstack can restore the original DataFrame if the index matches. Missing data in the stacked Series will appear as NaN in the unstacked DataFrame. This process is useful for converting data between wide and long formats in data analysis.

Practice

1. What does the stack() method do to a DataFrame in pandas?

easy

A. It moves columns into rows, making the data longer.

B. It moves rows into columns, making the data wider.

C. It sorts the DataFrame by index.

D. It removes duplicate rows from the DataFrame.

Solution

1. Step 1: Understand the stack operation

   The stack() method compresses columns into a single column, turning columns into rows.

2. Step 2: Compare with unstack

   Unlike unstack(), which moves rows to columns, stack() makes data longer by stacking columns.

3. Final Answer:

   It moves columns into rows, making the data longer. -> Option A

4. Quick Check:

   Stack = columns to rows [OK]

Hint: Stack compresses columns into rows, making data longer [OK]

Common Mistakes:

• Confusing stack with unstack direction
• Thinking stack sorts data
• Assuming stack removes duplicates

2. Which of the following is the correct syntax to unstack the last level of a MultiIndex DataFrame df?

easy

A. df.stack(axis=0)

B. df.stack(level=-1)

C. df.unstack(level=-1)

D. df.unstack(axis=1)

Solution

1. Step 1: Recall unstack syntax

   The unstack() method by default unstacks the last level of the index, and level=-1 explicitly specifies this.

2. Step 2: Differentiate from stack

   stack() is the opposite operation, so options A and B are incorrect. unstack() does not take an axis parameter, so df.unstack(axis=1) is invalid.

3. Final Answer:

   df.unstack(level=-1) -> Option C

4. Quick Check:

   Unstack last level = df.unstack(level=-1) [OK]

Hint: Use unstack(level=-1) to unstack last index level [OK]

Common Mistakes:

• Using stack instead of unstack
• Passing axis parameter to unstack
• Not specifying level when needed

3. Given the DataFrame df below:

import pandas as pd
d = {('A', 'x'): 1, ('A', 'y'): 2, ('B', 'x'): 3, ('B', 'y'): 4}
df = pd.Series(d).unstack()

What is the output of df.stack()?

medium

A. A DataFrame with columns 'A' and 'B' stacked

B. A Series with MultiIndex where second level is stacked into rows

C. A DataFrame with columns 'x' and 'y' unchanged

D. An error because stack cannot be applied here

Solution

1. Step 1: Understand the initial DataFrame

   The unstack() converts the Series with MultiIndex into a DataFrame with columns 'x' and 'y' and index 'A' and 'B'.

2. Step 2: Apply stack() on the DataFrame

   Calling stack() moves the columns 'x' and 'y' back into the inner level of the row index, producing a Series with MultiIndex.

3. Final Answer:

   A Series with MultiIndex where second level is stacked into rows -> Option B

4. Quick Check:

   Stack reverses unstack, making columns into row index [OK]

Hint: Stack reverses unstack, turning columns back into row index [OK]

Common Mistakes:

• Expecting a DataFrame output instead of Series
• Confusing which index level is stacked
• Thinking stack causes error here

4. You run the code below and get an error:

df = pd.DataFrame({
  'A': [1, 2],
  'B': [3, 4]
})
stacked = df.stack(level=1)

What is the cause of the error?

medium

A. The DataFrame index is missing.

B. Stack requires numeric data only.

C. Stack cannot be used on DataFrames, only Series.

D. DataFrame columns have no MultiIndex, so level=1 is invalid.

Solution

1. Step 1: Check DataFrame columns

   The DataFrame has simple columns 'A' and 'B', which are not a MultiIndex, so specifying level=1 is invalid.

2. Step 2: Understand stack level parameter

   The level parameter in stack() refers to the level of the columns MultiIndex to stack. Without MultiIndex columns, level=1 causes an error.

3. Final Answer:

   DataFrame columns have no MultiIndex, so level=1 is invalid. -> Option D

4. Quick Check:

   Stack level requires MultiIndex columns [OK]

Hint: Stack level needs MultiIndex columns; simple columns cause error [OK]

Common Mistakes:

• Assuming stack only works on Series
• Thinking stack requires numeric data
• Ignoring the column index type

5. You have a DataFrame df with a MultiIndex on rows (levels: 'City', 'Year') and columns as years (e.g., 2020, 2021). You want to reshape it so that the 'Year' level from rows becomes columns, and the columns years become a new row level. Which sequence of operations correctly achieves this?

hard

A. First unstack the 'Year' level from rows, then stack the columns.

B. First stack the columns, then unstack the 'City' level from rows.

C. Use only stack on the columns without unstacking rows.

D. Use only unstack on the 'City' level.

Solution

1. Step 1: Unstack the 'Year' level from rows

   Unstacking the 'Year' level moves it from the row index to columns, widening the DataFrame.

2. Step 2: Stack the columns

   Stacking the columns moves the columns years into a new row index level, making the data longer again but reshaped as desired.

3. Final Answer:

   First unstack the 'Year' level from rows, then stack the columns. -> Option A

4. Quick Check:

   Unstack rows then stack columns reshapes MultiIndex correctly [OK]

Hint: Unstack row level first, then stack columns to reshape MultiIndex [OK]

Common Mistakes:

• Stacking before unstacking causes wrong shape
• Unstacking wrong index level
• Ignoring the need to reshape both rows and columns