groupby() basics in Pandas - Step-by-Step Execution

Concept Flow - groupby() basics

Start with DataFrame

↓

Choose column(s) to group by

↓

Split data into groups

↓

Apply aggregation or function to each group

↓

Combine results into new DataFrame or Series

↓

Output grouped result

The groupby() process splits data by chosen columns, applies a function to each group, then combines the results.

Execution Sample

Pandas

import pandas as pd

data = {'Team': ['A', 'A', 'B', 'B'], 'Points': [10, 15, 10, 20]}
df = pd.DataFrame(data)
grouped = df.groupby('Team').sum()

This code groups rows by 'Team' and sums the 'Points' for each team.

Execution Table

Step	Action	Data State	Result
1	Create DataFrame	{'Team': ['A', 'A', 'B', 'B'], 'Points': [10, 15, 10, 20]}	4 rows with Teams and Points
2	Call groupby('Team')	DataFrame unchanged	Groups created: 'A' with 2 rows, 'B' with 2 rows
3	Apply sum() to groups	Groups split	Sum Points for 'A': 25, for 'B': 30
4	Combine results	Aggregated sums	New DataFrame with Teams as index and summed Points
5	Output grouped result	Final grouped DataFrame	Team A: 25, Team B: 30

💡 All groups processed and aggregated sums calculated

Variable Tracker

Variable	Start	After groupby	After sum	Final
df	Original DataFrame	Unchanged	Unchanged	Unchanged
grouped	Not defined	GroupBy object with groups 'A' and 'B'	DataFrame with sums per group	DataFrame with sums per group

Key Moments - 3 Insights

Why does groupby() not immediately show results?

groupby() creates a grouping object but does not compute results until an aggregation like sum() is called, as seen in steps 2 and 3 of the execution_table.

What happens if we group by a column but don't aggregate?

Without aggregation, groupby() returns a GroupBy object that holds groups but no summarized data, so no output is shown until an aggregation function is applied (step 3).

Why is the result indexed by the group column?

After aggregation, pandas uses the group column as the index to clearly show each group's result, as in step 4 where 'Team' becomes the index.

Visual Quiz - 3 Questions

Look at the execution_table at step 3, what is the sum of Points for group 'B'?

A 30

B 20

C 10

D 40

💡 Hint

Check the 'Result' column at step 3 where sums are calculated for each group.

At which step does the DataFrame change from original to aggregated sums?

A Step 2

B Step 3

C Step 4

D Step 5

💡 Hint

Look at the 'Data State' column to see when aggregation results are combined into a new DataFrame.

If we remove the sum() call, what type of object is 'grouped' after step 2?

A DataFrame

B GroupBy object

C Series

D List

💡 Hint

Refer to the 'Result' column at step 2 describing the groupby() output.

Concept Snapshot

pandas groupby() basics:
- Use df.groupby('col') to split data by 'col'
- Aggregation like sum(), mean() applies to each group
- Result is combined into new DataFrame or Series
- Group column becomes index in result
- groupby() alone returns a GroupBy object, no output until aggregation

Full Transcript

This lesson shows how pandas groupby() works step-by-step. First, a DataFrame is created with teams and points. Then groupby('Team') splits the data into groups by team. Next, sum() adds points within each group. Finally, results combine into a new DataFrame indexed by team. The key is groupby() creates groups but does not compute until aggregation is called. The output shows total points per team clearly. This helps summarize data by categories easily.

Practice

1. What does the groupby() function do in pandas?

easy

A. It splits the data into groups based on column values.

B. It sorts the data in ascending order.

C. It removes duplicate rows from the data.

D. It changes the data type of columns.

Solution

1. Step 1: Understand the purpose of groupby()

   The groupby() function is used to split data into groups based on the values in one or more columns.

2. Step 2: Compare with other options

   Sorting, removing duplicates, and changing data types are different operations and not what groupby() does.

3. Final Answer:

   It splits the data into groups based on column values. -> Option A

4. Quick Check:

   groupby() = split data into groups [OK]

Hint: Remember: groupby splits data by column values [OK]

Common Mistakes:

• Confusing groupby with sorting
• Thinking groupby removes duplicates
• Assuming groupby changes data types

2. Which of the following is the correct syntax to group a DataFrame df by the column 'Category'?

easy

A. df.group_by('Category')

B. df.groupby['Category']

C. df.groupby('Category')

D. df.groupby(Category)

Solution

1. Step 1: Recall the correct method name and syntax

   The correct method is groupby() called with parentheses and the column name as a string argument.

2. Step 2: Check each option

   df.groupby('Category') uses df.groupby('Category') which is correct. df.group_by('Category') uses wrong method name group_by. df.groupby['Category'] uses square brackets instead of parentheses. df.groupby(Category) misses quotes around the column name.

3. Final Answer:

   df.groupby('Category') -> Option C

4. Quick Check:

   Correct syntax uses parentheses and string column name [OK]

Hint: Use df.groupby('column_name') with parentheses and quotes [OK]

Common Mistakes:

• Using wrong method name like group_by
• Using square brackets instead of parentheses
• Not quoting the column name string

3. Given the DataFrame:

import pandas as pd
data = {'Team': ['A', 'A', 'B', 'B', 'C'], 'Points': [10, 15, 10, 5, 20]}
df = pd.DataFrame(data)
result = df.groupby('Team').sum()

What is the output of result?

medium

A. Team Points
A 25
B 15
C 20

B. {'Team': ['A', 'B', 'C'], 'Points': [25, 15, 20]}

C. Team Points
A 10
B 10
C 20

D. Team Points
A 15
B 5
C 20

Solution

1. Step 1: Group data by 'Team'

   The groups are: A with points 10 and 15, B with points 10 and 5, C with points 20.

2. Step 2: Sum points in each group

   Sum for A = 10 + 15 = 25, B = 10 + 5 = 15, C = 20.

3. Final Answer:

   Team Points A 25 B 15 C 20 -> Option A

4. Quick Check:

   Sum points per team = 25, 15, 20 [OK]

Hint: Sum points after groupby to get totals per group [OK]

Common Mistakes:

• Not summing points, just taking first value
• Mixing up group sums
• Confusing output format

4. What is wrong with this code snippet?

df = pd.DataFrame({'City': ['X', 'Y', 'X'], 'Sales': [100, 200, 150]})
result = df.groupby.City.sum()

medium

A. DataFrame is missing import statement.

B. The column name 'City' is misspelled.

C. The sum() function cannot be used after groupby.

D. You cannot use dot notation after groupby; it should be parentheses.

Solution

1. Step 1: Check groupby syntax

   The correct way to group by a column is df.groupby('City') with parentheses and quotes.

2. Step 2: Identify the error in dot notation

   Using dot notation like df.groupby.City is invalid because groupby is a method, not an attribute.

3. Final Answer:

   You cannot use dot notation after groupby; it should be parentheses. -> Option D

4. Quick Check:

   Use parentheses with groupby, not dot notation [OK]

Hint: Always use parentheses with groupby, not dot notation [OK]

Common Mistakes:

• Using dot notation instead of parentheses
• Misspelling column names
• Thinking sum() is invalid after groupby

5. You have a sales DataFrame:

import pandas as pd
data = {'Store': ['A', 'A', 'B', 'B', 'B'], 'Product': ['X', 'Y', 'X', 'Y', 'X'], 'Sales': [100, 150, 200, 250, 300]}
df = pd.DataFrame(data)

How do you find the total sales for each store and product combination?

hard

A. df.groupby('Product').sum('Store')

B. df.groupby(['Store', 'Product']).sum()

C. df.groupby('Store').sum('Product')

D. df.groupby(['Store']).sum(['Product'])

Solution

1. Step 1: Understand grouping by multiple columns

   To group by both 'Store' and 'Product', pass a list of column names to groupby().

2. Step 2: Use sum() to aggregate sales

   After grouping, use sum() to add sales values for each group.

3. Final Answer:

   df.groupby(['Store', 'Product']).sum() -> Option B

4. Quick Check:

   Group by multiple columns using list, then sum [OK]

Hint: Group by list of columns for multi-level grouping [OK]

Common Mistakes:

• Passing column names to sum() instead of groupby()
• Grouping by single column only
• Using wrong syntax for groupby