How to Use apply in pandas groupby for Custom Aggregations

Use apply with groupby in pandas to run a custom function on each group of data. This lets you perform complex operations that built-in aggregation methods can't handle. The syntax is df.groupby('column').apply(your_function) where your_function takes a DataFrame group and returns a result.

Syntax

The basic syntax for using apply with groupby is:

• df.groupby('column').apply(func)

Here, df is your DataFrame, 'column' is the column to group by, and func is a function you define that takes a DataFrame (each group) and returns a DataFrame, Series, or scalar.

This allows you to run any custom operation on each group.

python

df.groupby('column').apply(func)

Example

This example groups data by the 'Team' column and uses apply to calculate the range (max - min) of scores in each group.

python

import pandas as pd

data = {'Team': ['A', 'A', 'B', 'B', 'C', 'C'],
        'Score': [10, 15, 10, 20, 5, 7]}
df = pd.DataFrame(data)

def score_range(group):
    return group['Score'].max() - group['Score'].min()

result = df.groupby('Team').apply(score_range)
print(result)

Output

Team
A    5
B   10
C    2
dtype: int64

Common Pitfalls

Common mistakes when using apply with groupby include:

• Returning inconsistent types from the function, which can cause errors or unexpected output.
• Using apply when simpler aggregation methods like sum or mean would be faster.
• Not handling empty groups or missing data inside the function.

Always test your function on a single group before applying it to all groups.

python

import pandas as pd

data = {'Team': ['A', 'A', 'B', 'B'], 'Score': [10, 15, 10, 20]}
df = pd.DataFrame(data)

# Wrong: returns different types for groups

def inconsistent_func(group):
    if group.name == 'A':
        return group['Score'].sum()
    else:
        return group[['Score']]

# This will raise an error or produce confusing output
# df.groupby('Team').apply(inconsistent_func)  # Uncomment to see error

# Correct: always return scalar or Series

def consistent_func(group):
    return group['Score'].sum()

result = df.groupby('Team').apply(consistent_func)
print(result)

Output

Team
A    25
B    30
dtype: int64

Quick Reference

Tips for using apply with groupby:

• Use apply for custom functions that need the full group data.
• Return consistent types from your function for predictable results.
• For simple aggregations, prefer built-in methods like sum, mean, or agg for better performance.
• Test your function on a single group before applying it to all groups.

Key Takeaways

Use df.groupby('column').apply(func) to run custom functions on each group.

Ensure your function returns consistent output types for all groups.

Prefer built-in aggregation methods for simple calculations to improve speed.

Test your function on one group before applying it to the entire DataFrame.

apply allows flexible, complex operations beyond standard aggregations.