GroupBy performance considerations in Pandas - Mini Project: Build & Apply

GroupBy Performance Considerations

📖 Scenario: You work as a data analyst for a retail company. You have sales data for different products and stores. You want to find the total sales per store. But you also want to learn how to do this efficiently using pandas groupby.

🎯 Goal: Build a small pandas DataFrame with sales data, set a configuration for filtering, use groupby to calculate total sales per store efficiently, and print the result.

📋 What You'll Learn

Create a pandas DataFrame with exact sales data

Create a filter threshold variable

Use groupby on the DataFrame with the exact column names

Print the grouped result showing total sales per store

💡 Why This Matters

🌍 Real World

Retail companies often analyze sales data by store to understand performance and make decisions.

💼 Career

Data analysts and data scientists use pandas groupby to summarize and analyze data efficiently.

1

Create the sales DataFrame

Create a pandas DataFrame called sales_data with these exact columns and rows:
Store: ['Store A', 'Store B', 'Store A', 'Store C', 'Store B']
Product: ['Apples', 'Bananas', 'Oranges', 'Apples', 'Oranges']
Sales: [100, 150, 200, 130, 170]

Pandas

import pandas as pd
# Create the sales_data DataFrame with the exact data
# Your code here

Hint

Use pd.DataFrame with a dictionary where keys are column names and values are lists of data.

2

Set a sales filter threshold

Create a variable called min_sales and set it to 150. This will help filter stores with sales above this value later.

Pandas

import pandas as pd
sales_data = pd.DataFrame({
    'Store': ['Store A', 'Store B', 'Store A', 'Store C', 'Store B'],
    'Product': ['Apples', 'Bananas', 'Oranges', 'Apples', 'Oranges'],
    'Sales': [100, 150, 200, 130, 170]
})
# Create min_sales variable and set it to 150
# Your code here

Hint

Just assign the number 150 to the variable min_sales.

3

Group by Store and calculate total sales

Use groupby on sales_data by the 'Store' column. Then calculate the sum of 'Sales' for each store. Store the result in a variable called total_sales_per_store. Then filter total_sales_per_store to keep only stores with sales greater than or equal to min_sales.

Pandas

import pandas as pd
sales_data = pd.DataFrame({
    'Store': ['Store A', 'Store B', 'Store A', 'Store C', 'Store B'],
    'Product': ['Apples', 'Bananas', 'Oranges', 'Apples', 'Oranges'],
    'Sales': [100, 150, 200, 130, 170]
})
min_sales = 150
# Group sales_data by 'Store' and sum 'Sales'
# Filter stores with total sales >= min_sales
# Your code here

Hint

Use groupby('Store')['Sales'].sum() to get total sales per store. Then filter with total_sales_per_store[total_sales_per_store >= min_sales].

4

Print the total sales per store

Print the total_sales_per_store variable to see the total sales for each store that meets the sales threshold.

Pandas

import pandas as pd
sales_data = pd.DataFrame({
    'Store': ['Store A', 'Store B', 'Store A', 'Store C', 'Store B'],
    'Product': ['Apples', 'Bananas', 'Oranges', 'Apples', 'Oranges'],
    'Sales': [100, 150, 200, 130, 170]
})
min_sales = 150
total_sales_per_store = sales_data.groupby('Store')['Sales'].sum()
total_sales_per_store = total_sales_per_store[total_sales_per_store >= min_sales]
# Print the total_sales_per_store
# Your code here

Hint

Use print(total_sales_per_store) to display the result.

Practice

1. What is a key factor that affects the speed of pandas groupby operations?

easy

A. The type of operating system

B. The number of columns used to group the data

C. The font size of the code editor

D. The color of the data points

Solution

1. Step 1: Understand what affects groupby speed

   The more columns you group by, the more complex the operation becomes.

2. Step 2: Relate grouping columns to performance

   Grouping by fewer columns reduces computation and speeds up the process.

3. Final Answer:

   The number of columns used to group the data -> Option B

4. Quick Check:

   Grouping columns = performance impact [OK]

Hint: Fewer grouping columns usually mean faster groupby [OK]

Common Mistakes:

• Thinking system or editor settings affect groupby speed
• Ignoring the number of grouping columns
• Confusing data color or style with performance

2. Which of the following pandas groupby syntax examples is correct for grouping by column 'A' and calculating the mean of column 'B'?

easy

A. df.groupby('A')['B'].mean()

B. df.group_by('A').mean('B')

C. df.groupby('B')['A'].mean()

D. df.groupby('A').mean['B']

Solution

1. Step 1: Check correct groupby syntax

   The correct method is df.groupby('A')['B'].mean() to group by 'A' and get mean of 'B'.

2. Step 2: Identify syntax errors in other options

   Options A, B, and D have wrong method names or wrong column selections.

3. Final Answer:

   df.groupby('A')['B'].mean() -> Option A

4. Quick Check:

   Correct groupby syntax = df.groupby('A')['B'].mean() [OK]

Hint: Use df.groupby('col')['col2'].agg() format [OK]

Common Mistakes:

• Using group_by instead of groupby
• Swapping columns in groupby and aggregation
• Using wrong method call syntax

3. Given the code below, what will be the output?

import pandas as pd
df = pd.DataFrame({
  'Category': ['A', 'A', 'B', 'B', 'C'],
  'Value': [10, 20, 10, 30, 50]
})
result = df.groupby('Category').sum()
print(result)

medium

A. {'A': 30, 'B': 40, 'C': 50}

B. Category\nA 30\nB 40\nC 50

C. Value\nCategory\nA 30\nB 40\nC 50

D. Error due to missing aggregation

Solution

1. Step 1: Understand groupby sum output

   Grouping by 'Category' sums 'Value' for each group, resulting in sums 30, 40, and 50.

2. Step 2: Recognize pandas output format

   The output is a DataFrame with 'Value' as column and 'Category' as index, shown as:
   Value
   Category
   A 30
   B 40
   C 50

3. Final Answer:

   Value Category A 30 B 40 C 50 -> Option C

4. Quick Check:

   Group sum output = Value\nCategory\nA 30\nB 40\nC 50 [OK]

Hint: Groupby sum returns DataFrame with grouped index and summed columns [OK]

Common Mistakes:

• Expecting a dictionary instead of DataFrame
• Ignoring index and column labels in output
• Thinking sum needs explicit aggregation

4. What is wrong with this code snippet that groups by columns 'A' and 'B' and calculates the mean?

df.groupby(['A', 'B']).mean

medium

A. Mean cannot be used with multiple grouping columns

B. Wrong method name, should be avg()

C. Groupby should be called with a dictionary

D. Missing parentheses to call the mean function

Solution

1. Step 1: Check method call syntax

   The code uses mean without parentheses, so it returns a method object, not the result.

2. Step 2: Correct the syntax

   Adding parentheses mean() calls the function and returns the aggregated result.

3. Final Answer:

   Missing parentheses to call the mean function -> Option D

4. Quick Check:

   Function call needs () [OK]

Hint: Always add () to call aggregation functions [OK]

Common Mistakes:

• Forgetting parentheses after aggregation functions
• Using avg() instead of mean()
• Thinking groupby needs dict argument

5. You have a large DataFrame with columns 'City', 'Year', and 'Sales'. To improve groupby performance when calculating total sales per city, which approach is best?

hard

A. Group only by 'City' to reduce grouping complexity

B. Group by both 'City' and 'Year' to get detailed results

C. Sort the DataFrame by 'Sales' before grouping

D. Convert 'City' to string type before grouping

Solution

1. Step 1: Analyze grouping columns impact

   Grouping by fewer columns reduces the number of groups and speeds up computation.

2. Step 2: Choose grouping for total sales per city

   Since total sales per city is needed, grouping only by 'City' is sufficient and faster.

3. Final Answer:

   Group only by 'City' to reduce grouping complexity -> Option A

4. Quick Check:

   Less grouping columns = better performance [OK]

Hint: Group by only needed columns for faster results [OK]

Common Mistakes:

• Grouping by extra columns unnecessarily
• Sorting before grouping does not improve speed
• Changing data types without reason