Split-apply-combine mental model in Pandas - Step-by-Step Execution

Concept Flow - Split-apply-combine mental model

Start with DataFrame

↓

Split data into groups

↓

Apply function to each group

↓

Combine results into new DataFrame

↓

Output

The data is split into groups, a function is applied to each group, then results are combined into one output.

Execution Sample

Pandas

import pandas as pd

df = pd.DataFrame({'Team': ['A', 'A', 'B', 'B'], 'Points': [10, 15, 10, 20]})
result = df.groupby('Team')['Points'].mean()
print(result)

This code groups data by 'Team', calculates average 'Points' per team, and prints the result.

Execution Table

Step	Action	Data State	Result
1	Create DataFrame	df with 4 rows, columns Team and Points	DataFrame ready
2	Group by 'Team'	Groups: 'A' with 2 rows, 'B' with 2 rows	Groups formed
3	Apply mean to 'Points' in each group	Group A Points: [10, 15], Group B Points: [10, 20]	Means: A=12.5, B=15.0
4	Combine results	Series with index Team and mean Points	Output Series ready
5	Print result	Series displayed	Team A 12.5 B 15.0 Name: Points, dtype: float64

💡 All groups processed and combined, output produced

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	Final
df	None	DataFrame with 4 rows	Same DataFrame	Same DataFrame	Same DataFrame	Same DataFrame
groups	None	None	DataFrameGroupBy object with keys 'A' and 'B'	Same groups	Same groups	Same groups
result	None	None	None	Series with means per group	Same Series	Same Series

Key Moments - 3 Insights

Why do we split the data before applying the function?

Splitting groups data by a key so the function applies separately to each group, as shown in step 2 and 3 of the execution_table.

What does 'combine' mean in this model?

Combine means putting all group results back together into one structure, like the Series in step 4 and 5.

Is the original DataFrame changed after groupby?

No, the original DataFrame stays the same; groupby creates groups without modifying the original, as seen in variable_tracker for 'df'.

Visual Quiz - 3 Questions

Look at the execution_table, what is the mean Points for group 'A' after step 3?

A 12.5

B 15.0

C 10

D 20

💡 Hint

Check the 'Apply mean' row in execution_table step 3.

At which step are the groups actually formed?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Look at the 'Group by' action in execution_table.

If we changed the function from mean to sum, what would change in the execution_table?

A Groups would change

B DataFrame would have more rows

C Step 3 result values would be sums instead of means

D No change at all

💡 Hint

Focus on the 'Apply function' step in execution_table.

Concept Snapshot

Split-apply-combine model:
1. Split data into groups by key
2. Apply function to each group separately
3. Combine results into one output
Use pandas groupby() to do this easily
Example: df.groupby('key')['value'].mean()

Full Transcript

The split-apply-combine model means we start with a DataFrame, split it into groups by a key column, apply a function like mean to each group, then combine the results into a new output. In the example, we group by 'Team', calculate average 'Points' per team, and get a Series with these averages. The original DataFrame stays unchanged. This model helps analyze data by groups simply and clearly.