GroupBy with transform for normalization in Pandas

Introduction

We use groupby with transform to normalize data within groups. This helps compare values fairly inside each group.

You want to scale test scores within each class to see who did better compared to classmates.

You have sales data by region and want to compare sales performance normalized by each region's average.

You want to adjust employee salaries by department to see who earns more or less than their department average.

Syntax

Pandas

df['normalized'] = df.groupby('group_column')['value_column'].transform(lambda x: (x - x.mean()) / x.std())

Use groupby('group_column') to split data into groups.

transform() applies a function to each group and returns a result aligned with the original data.

Examples

Normalize scores within each class by subtracting the class mean and dividing by class standard deviation.

Pandas

df['score_norm'] = df.groupby('class')['score'].transform(lambda x: (x - x.mean()) / x.std())

Scale sales between 0 and 1 within each region using min-max normalization.

Pandas

df['sales_norm'] = df.groupby('region')['sales'].transform(lambda x: (x - x.min()) / (x.max() - x.min()))

Sample Program

This code creates a small table of classes and scores. It then normalizes scores inside each class group. The result shows normalized scores aligned with original rows.

Pandas

import pandas as pd

data = {'class': ['A', 'A', 'B', 'B', 'B'],
        'score': [80, 90, 70, 75, 85]}
df = pd.DataFrame(data)

df['score_norm'] = df.groupby('class')['score'].transform(lambda x: (x - x.mean()) / x.std())

print(df)

Important Notes

Normalization helps compare values fairly inside groups with different scales.

transform() keeps the original DataFrame shape, unlike apply() which can change it.

Summary

Use groupby to split data into groups.

transform() applies a function and returns results aligned with original data.

Normalization inside groups helps compare values fairly.