Split-apply-combine mental model in Pandas - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the 'split' step mean in the split-apply-combine model?

The 'split' step means dividing the data into groups based on some criteria, like grouping people by age or city.

beginner

Explain the 'apply' step in the split-apply-combine model.

The 'apply' step means doing some calculation or operation on each group separately, like finding the average score for each city.

beginner

What happens in the 'combine' step of the split-apply-combine model?

In the 'combine' step, the results from each group are put back together into one dataset, so you get a summary for all groups.

beginner

Which pandas function is commonly used to perform the split-apply-combine process?

The pandas function 'groupby()' is used to split data into groups, then you apply functions like mean(), sum(), or custom functions, and combine results.

beginner

Why is the split-apply-combine model useful in data analysis?

It helps to analyze data by groups easily, like finding patterns or summaries for each group without writing complex code.

What is the first step in the split-apply-combine model?

A Combine results from different datasets

B Apply a function to the whole dataset

C Split the data into groups

D Clean the data

The first step is to split the data into groups based on some criteria.

Which pandas method is mainly used for the split step?

A groupby()

B filter()

C merge()

D pivot()

The groupby() method splits data into groups for further analysis.

What does the 'apply' step do in the split-apply-combine model?

A Performs calculations on each group

B Splits data into groups

C Combines data from different sources

D Deletes missing data

The apply step performs calculations or operations on each group separately.

After applying functions to groups, what is the next step?

A Split again

B Visualize the data

C Sort the data

D Combine the results

The combine step puts the results from each group back together.

Which of these is NOT a typical function used in the apply step?

A mean()

B groupby()

C sum()

D custom function

groupby() is used to split data, not to apply calculations.

Describe the split-apply-combine mental model and how it helps in data analysis.

Think about how you might find average sales per store.

Explain how you would use pandas to find the average value of a column for each group in a dataset.

Remember the three steps: split, apply, combine.

Practice

1. What is the main idea behind the split-apply-combine mental model in pandas?

easy

A. Create new columns, rename them, then delete old columns

B. Sort data, filter rows, then plot results

C. Split data into groups, apply a function to each group, then combine results

D. Load data, clean missing values, then save to file

Solution

1. Step 1: Understand the split step

   The data is divided into groups based on some criteria using groupby().

2. Step 2: Apply and combine steps

   A function is applied to each group separately, then the results are combined into a new DataFrame.

3. Final Answer:

   Split data into groups, apply a function to each group, then combine results -> Option C

4. Quick Check:

   Split-apply-combine = Group, apply, combine [OK]

Hint: Remember: split groups, apply function, combine results [OK]

Common Mistakes:

• Confusing split-apply-combine with sorting or filtering
• Thinking it only applies to plotting or saving files
• Mixing up the order of steps

2. Which of the following is the correct syntax to group a DataFrame df by the column 'Category' and calculate the mean of each group?

easy

A. df.groupby('Category').mean()

B. df.mean().groupby('Category')

C. df.groupby.mean('Category')

D. df.groupby('Category').apply(mean)

Solution

1. Step 1: Identify correct groupby syntax

   The correct way to group by a column is df.groupby('Category').

2. Step 2: Apply aggregation function

   To get the mean of each group, use .mean() after grouping.

3. Final Answer:

   df.groupby('Category').mean() -> Option A

4. Quick Check:

   Groupby then mean = df.groupby('Category').mean() [OK]

Hint: Use df.groupby('col').mean() to get group means [OK]

Common Mistakes:

• Calling mean before groupby
• Using apply(mean) without importing mean function
• Wrong method chaining order

3. Given the DataFrame df below:

  Category  Value
0        A      5
1        B      3
2        A      7
3        B      2
4        C      4

What is the output of df.groupby('Category').sum()?

medium

A. Category as column with sums: A=5, B=3, C=4

B. Error because sum() cannot be applied

C. Sum of all values ignoring groups: 21

D. Category as index with sums: A=12, B=5, C=4

Solution

1. Step 1: Group data by 'Category'

   Rows are grouped by 'Category': A, B, and C groups.

2. Step 2: Sum values in each group

   Sum values for A: 5+7=12, B: 3+2=5, C: 4.

3. Final Answer:

   Category as index with sums: A=12, B=5, C=4 -> Option D

4. Quick Check:

   Sum per group = A:12, B:5, C:4 [OK]

Hint: Sum values after groupby to get totals per group [OK]

Common Mistakes:

• Summing all values ignoring groups
• Expecting category as a column instead of index
• Thinking sum() causes error

4. Consider this code snippet:

result = df.groupby('Type').agg({'Score': 'mean', 'Age': 'max'})

What is wrong if the code raises a KeyError?

medium

A. The aggregation functions 'mean' and 'max' are invalid

B. The DataFrame does not have a column named 'Type'

C. The groupby method cannot take a string argument

D. agg() requires a list, not a dictionary

Solution

1. Step 1: Understand KeyError cause

   A KeyError usually means the column used for grouping does not exist in the DataFrame.

2. Step 2: Check groupby argument and agg usage

   Using a string in groupby and a dictionary in agg is correct syntax.

3. Final Answer:

   The DataFrame does not have a column named 'Type' -> Option B

4. Quick Check:

   KeyError means missing column [OK]

Hint: Check column names before groupby to avoid KeyError [OK]

Common Mistakes:

• Assuming aggregation functions are invalid
• Thinking groupby can't take string columns
• Misunderstanding agg() argument types

5. You have a DataFrame df with columns 'Team', 'Player', and 'Points'. You want to find the player with the highest points in each team. Which code correctly uses the split-apply-combine model to achieve this?

hard

A. df.groupby('Team').apply(lambda g: g[g['Points'] == g['Points'].max()])

B. df.groupby('Team')['Points'].max()

C. df.groupby('Player')['Points'].sum()

D. df.groupby('Team').mean()['Points']

Solution

1. Step 1: Understand the goal

   We want the player with the highest points per team, not just the max points value.

2. Step 2: Use groupby and apply to filter rows

   Using apply with a lambda filters each group to rows where 'Points' equals the group's max.

3. Step 3: Check other options

   df.groupby('Team')['Points'].max() returns max points per team but not the player. df.groupby('Player')['Points'].sum() groups by player, not team. df.groupby('Team').mean()['Points'] returns mean points, not max player.

4. Final Answer:

   df.groupby('Team').apply(lambda g: g[g['Points'] == g['Points'].max()]) -> Option A

5. Quick Check:

   Use apply with filter to get max player per group [OK]

Hint: Use groupby + apply with filter to get max per group row [OK]

Common Mistakes:

• Using max() alone returns only values, not rows
• Grouping by wrong column
• Using mean() instead of max()