pivot_table() for summarization in Pandas

Introduction

pivot_table() helps you quickly summarize data by grouping and calculating statistics like sums or averages.

You want to see total sales by product and region.

You need the average score of students by class and subject.

You want to count how many times each category appears in your data.

You want to compare data across two or more categories easily.

Syntax

Pandas

pandas.pivot_table(data, values=None, index=None, columns=None, aggfunc='mean', fill_value=None)

data: your DataFrame.

values: column(s) to aggregate.

index: rows to group by.

columns: columns to group by.

aggfunc: function like 'sum', 'mean', or 'count'.

Examples

Sum sales for each product.

Pandas

pd.pivot_table(df, values='Sales', index='Product', aggfunc='sum')

Average score by class and subject.

Pandas

pd.pivot_table(df, values='Score', index='Class', columns='Subject', aggfunc='mean')

Count how many rows per category.

Pandas

pd.pivot_table(df, index='Category', aggfunc='size')

Sample Program

This code creates a table showing total sales for each product split by region. Missing values are filled with 0.

Pandas

import pandas as pd

data = {
    'Product': ['Apple', 'Apple', 'Banana', 'Banana', 'Carrot', 'Carrot'],
    'Region': ['East', 'West', 'East', 'West', 'East', 'West'],
    'Sales': [100, 150, 200, 250, 300, 350]
}

df = pd.DataFrame(data)

pivot = pd.pivot_table(df, values='Sales', index='Product', columns='Region', aggfunc='sum', fill_value=0)
print(pivot)

Important Notes

Use fill_value to replace missing values with a number like 0.

You can use multiple aggregation functions by passing a list to aggfunc.

pivot_table returns a DataFrame that is easy to read and analyze.

Summary

pivot_table() groups data by rows and columns and calculates summary stats.

It is useful for quick data summaries like sums, averages, or counts.

You can customize grouping and aggregation easily with its parameters.

Practice

1.

What does the pivot_table() function in pandas primarily do?

easy

A. It merges two dataframes based on a key column.

B. It sorts data in ascending or descending order.

C. It summarizes data by grouping rows and columns and calculating statistics.

D. It filters rows based on conditions.

Solution

1. Step 1: Understand the purpose of pivot_table()

   The pivot_table() function groups data by specified rows and columns and calculates summary statistics like mean, sum, or count.

2. Step 2: Compare with other options

   Sorting, filtering, and merging are different pandas operations, not the main purpose of pivot_table().

3. Final Answer:

   It summarizes data by grouping rows and columns and calculating statistics. -> Option C

4. Quick Check:

   pivot_table() = summary by groups [OK]

Hint: pivot_table groups and summarizes data quickly [OK]

Common Mistakes:

• Confusing pivot_table with sorting or filtering
• Thinking pivot_table merges dataframes
• Assuming pivot_table only reshapes data without aggregation

2.

Which of the following is the correct syntax to create a pivot table that calculates the mean of column 'Sales' grouped by 'Region'?

import pandas as pd
df = pd.DataFrame({
    'Region': ['East', 'West', 'East', 'West'],
    'Sales': [100, 200, 150, 300]
})

# Which line is correct?

easy

A. pd.pivot_table(df, index='Region', values='Sales', aggfunc='mean')

B. pd.pivot_table(df, columns='Region', values='Sales', aggfunc='sum')

C. pd.pivot_table(df, index='Sales', values='Region', aggfunc='mean')

D. pd.pivot_table(df, values='Region', index='Sales', aggfunc='count')

Solution

1. Step 1: Identify grouping and aggregation

   We want to group by 'Region' (index) and calculate mean of 'Sales' (values) using aggfunc='mean'.

2. Step 2: Check each option

   pd.pivot_table(df, index='Region', values='Sales', aggfunc='mean') matches the requirement exactly. Options B, C, and D either swap index and values or use wrong aggfunc.

3. Final Answer:

   pd.pivot_table(df, index='Region', values='Sales', aggfunc='mean') -> Option A

4. Quick Check:

   Group by Region, mean Sales = pd.pivot_table(df, index='Region', values='Sales', aggfunc='mean') [OK]

Hint: index=group, values=summary column, aggfunc=stat [OK]

Common Mistakes:

• Swapping index and values parameters
• Using wrong aggregation function
• Confusing columns parameter with index

3.

Given the following code, what is the output?

import pandas as pd
df = pd.DataFrame({
    'Category': ['A', 'A', 'B', 'B'],
    'Type': ['X', 'Y', 'X', 'Y'],
    'Value': [10, 20, 30, 40]
})
pivot = pd.pivot_table(df, index='Category', columns='Type', values='Value', aggfunc='sum')
print(pivot)

medium

A. Category X Y
A 10 20
B 30 40

B. Type X Y
Category
A 10.0 20.0
B 30.0 40.0

C. Type X Y
Category
A 20.0 10.0
B 40.0 30.0

D. Type X Y
Category
A 30.0 40.0
B 10.0 20.0

Solution

1. Step 1: Understand pivot_table parameters

   Grouping by 'Category' as rows, 'Type' as columns, summing 'Value'.

2. Step 2: Calculate sums for each group

   For Category A and Type X: 10, Type Y: 20; For B and X: 30, Y: 40.

3. Final Answer:

   Type X Y Category A 10.0 20.0 B 30.0 40.0 -> Option B

4. Quick Check:

   Sum by Category and Type = Type X Y Category A 10.0 20.0 B 30.0 40.0 [OK]

Hint: Sum values by row index and column labels [OK]

Common Mistakes:

• Mixing up rows and columns in output
• Confusing sum with mean aggregation
• Ignoring float formatting in output

4.

What is wrong with this pivot_table code?

import pandas as pd
df = pd.DataFrame({
    'Group': ['G1', 'G1', 'G2'],
    'Score': [5, 10, 15]
})
pivot = pd.pivot_table(df, index='Group', values='Score', aggfunc='average')
print(pivot)

medium

A. The aggfunc 'average' is invalid; should be 'mean'.

B. The 'values' parameter should be 'Group' instead of 'Score'.

C. The 'index' parameter should be 'Score' instead of 'Group'.

D. The DataFrame is missing a column for aggregation.

Solution

1. Step 1: Check aggfunc parameter

   The aggfunc parameter must be a valid aggregation function like 'mean', 'sum', or 'count'. 'average' is not recognized by pandas.

2. Step 2: Verify other parameters

   Index='Group' and values='Score' are correct for grouping and aggregation.

3. Final Answer:

   The aggfunc 'average' is invalid; should be 'mean'. -> Option A

4. Quick Check:

   aggfunc='mean' not 'average' [OK]

Hint: Use 'mean' not 'average' for aggregation [OK]

Common Mistakes:

• Using 'average' instead of 'mean' in aggfunc
• Swapping index and values parameters
• Assuming aggfunc accepts any descriptive name

5.

You have a sales dataset with columns 'Store', 'Product', and 'Revenue'. You want to create a pivot table that shows the total revenue for each store and product combination, but if a combination is missing, it should show 0 instead of NaN. Which code correctly achieves this?

hard

A. pd.pivot_table(df, index='Store', columns='Product', values='Revenue', aggfunc='mean', fill_value=0)

B. pd.pivot_table(df, index='Store', columns='Product', values='Revenue', aggfunc='sum').fillna('0')

C. pd.pivot_table(df, index='Product', columns='Store', values='Revenue', aggfunc='sum', fill_value=null)

D. pd.pivot_table(df, index='Store', columns='Product', values='Revenue', aggfunc='sum', fill_value=0)

Solution

1. Step 1: Identify grouping and aggregation

   We want total revenue (sum) grouped by Store (rows) and Product (columns).

2. Step 2: Handle missing combinations

   Use fill_value=0 in pivot_table to replace NaN with 0 for missing pairs.

3. Step 3: Check each option

   pd.pivot_table(df, index='Store', columns='Product', values='Revenue', aggfunc='sum', fill_value=0) uses correct index, columns, aggfunc, and fill_value=0. pd.pivot_table(df, index='Store', columns='Product', values='Revenue', aggfunc='sum').fillna('0') fills NaN after pivot but with string '0' (wrong type). pd.pivot_table(df, index='Store', columns='Product', values='Revenue', aggfunc='mean', fill_value=0) uses mean instead of sum. pd.pivot_table(df, index='Product', columns='Store', values='Revenue', aggfunc='sum', fill_value=null) swaps index and columns and fill_value=null.

4. Final Answer:

   pd.pivot_table(df, index='Store', columns='Product', values='Revenue', aggfunc='sum', fill_value=0) -> Option D

5. Quick Check:

   sum + fill_value=0 for missing = pd.pivot_table(df, index='Store', columns='Product', values='Revenue', aggfunc='sum', fill_value=0) [OK]

Hint: Use fill_value=0 to replace NaN in pivot_table [OK]

Common Mistakes:

• Using fillna('0') which creates string values
• Using mean instead of sum for total revenue
• Swapping index and columns parameters