filter() for group-level filtering in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of group filtering with filter()

What is the output DataFrame after applying filter() to keep groups with mean value greater than 3?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({
    'group': ['A', 'A', 'B', 'B', 'C', 'C'],
    'value': [2, 4, 5, 6, 1, 2]
})
result = df.groupby('group').filter(lambda x: x['value'].mean() > 3)
print(result)

A

  group  value
0     A      2
1     A      4
2     B      5
3     B      6

B

  group  value
2     B      5
3     B      6

C

  group  value
0     A      2
1     A      4
2     B      5
3     B      6
4     C      1
5     C      2

D

Empty DataFrame
Columns: [group, value]
Index: []

💡 Hint

Calculate the mean of 'value' for each group and keep groups where mean > 3.

Group 'B' has mean (5+6)/2 = 5.5 > 3, so only group 'B' rows remain after filtering.

❓ data_output

intermediate

Number of groups after filtering

After filtering groups with sum of 'score' >= 10, how many groups remain?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({
    'team': ['X', 'X', 'Y', 'Y', 'Z', 'Z'],
    'score': [4, 7, 3, 2, 6, 5]
})
filtered = df.groupby('team').filter(lambda g: g['score'].sum() >= 10)
num_groups = filtered['team'].nunique()
print(num_groups)

A 0

B 1

C 3

D 2

💡 Hint

Sum scores per team and count teams with sum >= 10.

Team X sum is 11, team Y sum is 5, team Z sum is 11. Teams X and Z remain, so 2 groups.

🔧 Debug

advanced

Identify the error in group filtering code

What error does this code raise when filtering groups with mean > 2?

df.groupby('cat').filter(lambda x: x['val'].mean > 2)

Data Analysis Python

import pandas as pd

df = pd.DataFrame({'cat': ['a', 'a', 'b'], 'val': [1, 3, 4]})
result = df.groupby('cat').filter(lambda x: x['val'].mean > 2)

A No error, outputs filtered DataFrame

B AttributeError: 'DataFrameGroupBy' object has no attribute 'filter'

C TypeError: 'method' object is not callable

D SyntaxError: invalid syntax

💡 Hint

Check if mean is called as a method or accessed as an attribute.

mean is a method and must be called with parentheses (). Missing () causes a method object to be returned, which is not callable in comparison.

🚀 Application

advanced

Filter groups with more than 2 rows and average value below 5

Which option correctly filters groups with more than 2 rows and average 'score' less than 5?

Data Analysis Python

import pandas as pd

df = pd.DataFrame({
    'group': ['G1', 'G1', 'G1', 'G2', 'G2', 'G3', 'G3', 'G3'],
    'score': [4, 3, 2, 6, 7, 1, 2, 3]
})

A df.groupby('group').filter(lambda g: len(g) > 2 and g['score'].mean() < 5)

B df.groupby('group').filter(lambda g: len(g) >= 2 and g['score'].mean() <= 5)

C df.groupby('group').filter(lambda g: len(g) > 3 and g['score'].mean() < 5)

D df.groupby('group').filter(lambda g: len(g) > 2 or g['score'].mean() < 5)

💡 Hint

Use both conditions with 'and' and strict inequalities as stated.

Only groups with more than 2 rows and mean score less than 5 are kept. G1 has 3 rows and mean 3, G3 has 3 rows and mean 2, G2 has 2 rows so excluded.

🧠 Conceptual

expert

Understanding filter() behavior on groupby objects

Which statement about filter() on a pandas GroupBy object is TRUE?

A filter() returns a DataFrame containing only groups where the filter function returns True for the entire group.

B filter() modifies the original DataFrame in place by removing groups that do not meet the condition.

C filter() applies the function to each row individually and returns rows where the function returns True.

D filter() returns a Series with group names as index and boolean values indicating if group passed the filter.

💡 Hint

Think about how filter works on groups, not individual rows.

filter() applies the function to each group and keeps groups where the function returns True. It returns a filtered DataFrame, not a Series or in-place modification.

Practice

1. What does the filter() function do when used with groupby in pandas?

easy

A. Filters individual rows inside groups without removing groups

B. Keeps or removes entire groups based on a condition function

C. Sorts groups by their size

D. Aggregates data by summing values in each group

Solution

1. Step 1: Understand filter() with groupby

   The filter() function applies a condition to each group and keeps or removes the whole group based on True or False.

2. Step 2: Differentiate from other functions

   Unlike aggregation or sorting, filter() does not change group content but decides if the group stays or goes.

3. Final Answer:

   Keeps or removes entire groups based on a condition function -> Option B

4. Quick Check:

   filter() = group-level keep/remove [OK]

Hint: filter() keeps or drops whole groups by condition [OK]

Common Mistakes:

• Thinking filter() works on individual rows only
• Confusing filter() with aggregation functions
• Assuming filter() sorts groups

2. Which of the following is the correct syntax to filter groups in a pandas DataFrame df grouped by column 'Category' where the group size is greater than 3?

easy

A. df.groupby('Category').filter(lambda x: len(x) > 3)

B. df.filter(groupby='Category', condition=lambda x: len(x) > 3)

C. df.groupby('Category').apply(lambda x: len(x) > 3)

D. df.groupby('Category').select(lambda x: len(x) > 3)

Solution

1. Step 1: Recall filter() syntax with groupby

   The correct syntax is df.groupby('Category').filter(function) where the function returns True or False per group.

2. Step 2: Check each option

   df.groupby('Category').filter(lambda x: len(x) > 3) matches the correct syntax. Options A, C, and D use invalid methods or parameters.

3. Final Answer:

   df.groupby('Category').filter(lambda x: len(x) > 3) -> Option A

4. Quick Check:

   Correct syntax uses groupby().filter() [OK]

Hint: Use groupby().filter(lambda x: condition) syntax [OK]

Common Mistakes:

• Using filter() directly on DataFrame without groupby
• Using apply() instead of filter() for filtering groups
• Passing wrong parameters to filter()

3. Given the following code, what will be the output?

import pandas as pd
data = {'Team': ['A', 'A', 'B', 'B', 'B', 'C'], 'Points': [10, 15, 5, 7, 8, 20]}
df = pd.DataFrame(data)
filtered = df.groupby('Team').filter(lambda x: x['Points'].mean() > 10)
print(filtered)

medium

A. Rows where Team is 'A' or 'C' only

B. Rows where Team is 'B' only

C. All rows from all teams

D. Empty DataFrame

Solution

1. Step 1: Calculate mean Points per Team

   Team A mean = (10+15)/2 = 12.5, Team B mean = (5+7+8)/3 = 6.67, Team C mean = 20

2. Step 2: Apply filter condition

   Keep groups with mean Points > 10, so Teams A and C remain, Team B is removed.

3. Final Answer:

   Rows where Team is 'A' or 'C' only -> Option A

4. Quick Check:

   Mean > 10 keeps A and C [OK]

Hint: Calculate group mean, keep groups meeting condition [OK]

Common Mistakes:

• Forgetting to calculate mean per group
• Assuming filter keeps rows, not groups
• Confusing group means with total sums

4. Identify the error in this code snippet that tries to filter groups with sum of 'Sales' greater than 100:

df.groupby('Region').filter(lambda x: x['Sales'].sum > 100)

medium

A. Lambda function should return False instead of True

B. filter() cannot be used with groupby

C. Missing parentheses after sum method call

D. Wrong column name 'Sales'

Solution

1. Step 1: Check the lambda function syntax

   The code uses x['Sales'].sum which is a method, but it lacks parentheses to call it.

2. Step 2: Correct the method call

   It should be x['Sales'].sum() to get the sum value for comparison.

3. Final Answer:

   Missing parentheses after sum method call -> Option C

4. Quick Check:

   sum() needs parentheses to execute [OK]

Hint: Remember to call sum() with parentheses [OK]

Common Mistakes:

• Using sum without parentheses
• Thinking filter() is invalid with groupby
• Assuming wrong column name causes error

5. You have a sales DataFrame df with columns 'Store', 'Month', and 'Revenue'. You want to keep only stores where the total revenue in any month exceeds 5000. Which code correctly filters these stores using filter()?

hard

A. df.groupby('Store').filter(lambda g: g['Revenue'].mean() > 5000)

B. df.groupby('Store').filter(lambda g: g['Revenue'].sum() > 5000)

C. df.groupby('Month').filter(lambda g: g['Revenue'].max() > 5000)

D. df.groupby('Store').filter(lambda g: g['Revenue'].max() > 5000)

Solution

1. Step 1: Understand the condition

   We want stores where revenue in any month exceeds 5000, so check max revenue per store.

2. Step 2: Analyze each option

   df.groupby('Store').filter(lambda g: g['Revenue'].max() > 5000) uses max() per store, matching the condition. df.groupby('Store').filter(lambda g: g['Revenue'].sum() > 5000) sums all revenue, which is different. df.groupby('Month').filter(lambda g: g['Revenue'].max() > 5000) groups by Month, not Store. df.groupby('Store').filter(lambda g: g['Revenue'].mean() > 5000) uses mean, which is not the condition.

3. Final Answer:

   df.groupby('Store').filter(lambda g: g['Revenue'].max() > 5000) -> Option D

4. Quick Check:

   max() per store matches condition [OK]

Hint: Use max() to check if any month exceeds threshold [OK]

Common Mistakes:

• Using sum() instead of max() for condition
• Grouping by wrong column
• Using mean() which may not meet condition