transform() for group-level operations in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of transform() with group mean

What is the output of the following code snippet using transform() to calculate the group mean?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({'Team': ['A', 'A', 'B', 'B', 'B'], 'Points': [10, 20, 30, 40, 50]})
df['MeanPoints'] = df.groupby('Team')['Points'].transform('mean')
print(df)

A {'Team': ['A', 'A', 'B', 'B', 'B'], 'Points': [10, 20, 30, 40, 50], 'MeanPoints': [10.0, 20.0, 30.0, 40.0, 50.0]}

B {'Team': ['A', 'A', 'B', 'B', 'B'], 'Points': [10, 20, 30, 40, 50], 'MeanPoints': [15.0, 15.0, 40.0, 40.0, 40.0]}

C {'Team': ['A', 'A', 'B', 'B', 'B'], 'Points': [10, 20, 30, 40, 50], 'MeanPoints': [15.0, 20.0, 40.0, 40.0, 50.0]}

D {'Team': ['A', 'A', 'B', 'B', 'B'], 'Points': [10, 20, 30, 40, 50], 'MeanPoints': [10.0, 10.0, 30.0, 30.0, 30.0]}

💡 Hint

Remember, transform() returns a series with the same size as the original, repeating the group statistic for each member.

The transform('mean') calculates the mean of each group and assigns that mean value to every row in the group. For team A, mean is (10+20)/2=15. For team B, mean is (30+40+50)/3=40.

❓ data_output

intermediate

Number of items after transform()

After applying transform() on a grouped DataFrame, how many rows does the resulting Series have compared to the original DataFrame?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({'Category': ['X', 'X', 'Y', 'Y', 'Y'], 'Value': [5, 10, 15, 20, 25]})
result = df.groupby('Category')['Value'].transform('max')
print(len(result))

A 5

B 2

C 3

D 1

💡 Hint

Think about whether transform() changes the number of rows or just the values.

transform() returns a Series with the same length as the original DataFrame, so the length remains 5.

🔧 Debug

advanced

Identify the error in transform() usage

What error does the following code raise?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({'Group': ['G1', 'G1', 'G2'], 'Score': [1, 2, 3]})
df['Result'] = df.groupby('Group')['Score'].transform(lambda x: x + x.shift())
print(df)

A TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

B KeyError: 'Score'

C ValueError: Length of values does not match length of index

D No error, prints DataFrame with NaN in first row of each group

💡 Hint

Check what shift() does inside each group and how it affects the addition.

The shift() moves values down by one within each group, causing the first row to have NaN. Adding NaN to a number results in NaN, but no error occurs.

🚀 Application

advanced

Using transform() to normalize data by group

You want to normalize the 'Sales' column within each 'Region' by subtracting the group mean and dividing by the group standard deviation. Which code snippet correctly does this using transform()?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({'Region': ['East', 'East', 'West', 'West', 'West'], 'Sales': [100, 150, 200, 250, 300]})

A df['Normalized'] = df.groupby('Region')['Sales'].transform(lambda x: (x - x.mean()) / x.std())

B df['Normalized'] = df.groupby('Region')['Sales'].apply(lambda x: (x - x.mean()) / x.std())

C df['Normalized'] = df['Sales'] / df.groupby('Region')['Sales'].transform('mean')

D df['Normalized'] = (df['Sales'] - df.groupby('Region')['Sales'].transform('mean')) / df.groupby('Region')['Sales'].transform('std')

💡 Hint

Remember transform() returns a Series aligned with the original DataFrame, while apply() may not.

Option A uses transform() with a lambda to normalize each group, returning a Series with the same size as the original DataFrame. Option A is correct mathematically but less flexible for complex functions. Option A only divides by mean, not normalizing. Option A uses apply() which returns a grouped object, not aligned with original DataFrame.

🧠 Conceptual

expert

Why use transform() instead of apply() for group-level operations?

Which statement best explains why transform() is preferred over apply() when you want to add a group-level statistic as a new column to the original DataFrame?

A <code>apply()</code> always returns a scalar value for each group, so it cannot be used to create new columns.

B <code>transform()</code> modifies the original DataFrame in place, while <code>apply()</code> creates a copy.

C <code>transform()</code> returns a Series with the same index and length as the original DataFrame, allowing direct assignment as a new column.

D <code>apply()</code> is slower because it does not support lambda functions, unlike <code>transform()</code>.

💡 Hint

Think about the shape and alignment of the output from both methods.

transform() returns a Series aligned with the original DataFrame's index and length, so it can be assigned as a new column easily. apply() returns aggregated results per group, often with fewer rows, making direct assignment impossible without extra steps.

Practice

1. What does the transform() function do when used on a grouped DataFrame in pandas?

easy

A. Applies a function to each group and returns a result with the same size as the original DataFrame

B. Aggregates each group into a single summary value, reducing the DataFrame size

C. Filters groups based on a condition and removes others

D. Sorts the groups in ascending order

Solution

1. Step 1: Understand the purpose of transform()

   The transform() function applies a function to each group but keeps the original DataFrame size by returning a transformed value for each row.

2. Step 2: Compare with other group operations

   Unlike aggregation which reduces group size, transform() returns a result aligned with the original data rows.

3. Final Answer:

   Applies a function to each group and returns a result with the same size as the original DataFrame -> Option A

4. Quick Check:

   transform() keeps original size [OK]

Hint: Remember: transform keeps row count same as original [OK]

Common Mistakes:

• Confusing transform with aggregate which reduces size
• Thinking transform filters groups
• Assuming transform sorts data

2. Which of the following is the correct syntax to apply transform() on a grouped DataFrame df grouped by column 'A' to calculate the mean of column 'B' for each group?

easy

A. df.transform('mean').groupby('A')['B']

B. df.groupby('A')['B'].transform('mean')

C. df['B'].groupby('A').transform('mean')

D. df.groupby('B')['A'].transform('mean')

Solution

1. Step 1: Identify grouping and column selection

   We group df by column 'A' and want to transform column 'B' by calculating its mean per group.

2. Step 2: Check syntax correctness

   The correct syntax is df.groupby('A')['B'].transform('mean') which applies mean to each group of 'B' and returns a Series aligned with original rows.

3. Final Answer:

   df.groupby('A')['B'].transform('mean') -> Option B

4. Quick Check:

   Group by 'A', transform 'B' with mean [OK]

Hint: Group first, select column, then transform [OK]

Common Mistakes:

• Applying transform before groupby
• Grouping by wrong column
• Selecting wrong column for transform

3. Given the DataFrame df below:

   Team  Score
0  A     10
1  A     20
2  B     30
3  B     40
4  B     50

What is the output of df.groupby('Team')['Score'].transform(lambda x: x - x.mean())?

medium

A. [ -5.0, 5.0, -10.0, 0.0, 10.0 ]

B. [ 10, 20, 30, 40, 50 ]

C. [ 0.0, 0.0, 0.0, 0.0, 0.0 ]

D. [ 15.0, 25.0, 40.0, 50.0, 60.0 ]

Solution

1. Step 1: Calculate group means

   Group A scores: 10, 20 -> mean = 15
   Group B scores: 30, 40, 50 -> mean = 40

2. Step 2: Subtract group mean from each score

   For group A: 10-15 = -5, 20-15 = 5
   For group B: 30-40 = -10, 40-40 = 0, 50-40 = 10

3. Final Answer:

   [-5.0, 5.0, -10.0, 0.0, 10.0] -> Option A

4. Quick Check:

   Subtract mean per group [OK]

Hint: Subtract group mean from each value using transform [OK]

Common Mistakes:

• Calculating overall mean instead of group mean
• Using aggregate instead of transform
• Misaligning output length

4. Consider this code snippet:

df.groupby('Category')['Value'].transform(np.sum)

It raises an error: NameError: name 'np' is not defined. How can you fix it?

medium

A. Use df.transform(np.sum).groupby('Category')['Value']

B. Replace np.sum with 'sum'

C. Add import numpy as np before the code

D. Change transform to aggregate

Solution

1. Step 1: Identify cause of error

   The error NameError: name 'np' is not defined means numpy was not imported but used as np.

2. Step 2: Fix by importing numpy

   Add import numpy as np at the start of the script to define np.

3. Final Answer:

   Add import numpy as np before the code -> Option C

4. Quick Check:

   Import libraries before use [OK]

Hint: Always import libraries before using their functions [OK]

Common Mistakes:

• Trying to use transform before groupby
• Misplacing groupby and transform order
• Replacing transform with aggregate without reason

5. You have a DataFrame df with columns 'Store', 'Sales', and some missing 'Sales' values. You want to fill missing 'Sales' with the average sales of the respective 'Store'. Which code correctly uses transform() to do this?

hard

A. df['Sales'] = df.groupby('Store')['Sales'].mean().fillna(df['Sales'])

B. df['Sales'] = df.groupby('Store')['Sales'].fillna('mean')

C. df['Sales'] = df.groupby('Store')['Sales'].transform('mean').fillna(df['Sales'])

D. df['Sales'] = df['Sales'].fillna(df.groupby('Store')['Sales'].transform('mean'))

Solution

1. Step 1: Understand the goal

   We want to fill missing 'Sales' values with the average sales per 'Store'. The average per store must align with each row.

2. Step 2: Use transform to get group means aligned to rows

   df.groupby('Store')['Sales'].transform('mean') returns a Series with the mean sales per store repeated for each row.

3. Step 3: Fill missing values with these means

   Use fillna() on 'Sales' column with the transformed means to fill missing values correctly.

4. Final Answer:

   df['Sales'] = df['Sales'].fillna(df.groupby('Store')['Sales'].transform('mean')) -> Option D

5. Quick Check:

   Fill missing with group mean using transform [OK]

Hint: Use transform to align group means for filling missing data [OK]

Common Mistakes:

• Using aggregate mean which returns smaller Series
• Filling missing before grouping
• Misusing fillna with wrong data shape