Why window functions matter in Pandas - Challenge Your Understanding

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of rolling mean with window size 3

What is the output of the following code that calculates a rolling mean with a window size of 3 on a pandas Series?

Pandas

import pandas as pd
s = pd.Series([10, 20, 30, 40, 50])
result = s.rolling(window=3).mean()
print(result.tolist())

A [nan, 15.0, 25.0, 35.0, 45.0]

B [10.0, 20.0, 30.0, 40.0, 50.0]

C [nan, nan, 20.0, 30.0, 40.0]

D [nan, nan, nan, 30.0, 40.0]

💡 Hint

Think about how rolling mean calculates the average of the current and previous values within the window.

The rolling mean with window=3 calculates the average of the current and previous two values. The first two positions have insufficient data, so they are NaN. Starting from the third value, the mean is computed over the last three values.

❓ data_output

intermediate

Result of cumulative sum grouped by category

Given the DataFrame below, what is the output of the cumulative sum of 'value' grouped by 'category'?

Pandas

import pandas as pd
df = pd.DataFrame({'category': ['A', 'A', 'B', 'B', 'A'], 'value': [1, 2, 3, 4, 5]})
df['cum_sum'] = df.groupby('category')['value'].cumsum()
print(df)

A

  category  value  cum_sum
0        A      1        1
1        A      2        3
2        B      3        3
3        B      4        7
4        A      5        8

B

  category  value  cum_sum
0        A      1        1
1        A      2        2
2        B      3        3
3        B      4        4
4        A      5        5

C

  category  value  cum_sum
0        A      1        1
1        A      2        3
2        B      3        3
3        B      4        4
4        A      5        5

D

  category  value  cum_sum
0        A      1        1
1        A      2        2
2        B      3        7
3        B      4        7
4        A      5        8

💡 Hint

Cumulative sum restarts for each group and adds values in order.

The cumulative sum adds values within each category group. For category A: 1, then 1+2=3, then 3+5=8. For category B: 3, then 3+4=7.

❓ visualization

advanced

Visualizing rolling average impact

Which plot correctly shows the original data and its rolling average with window size 4?

Pandas

import pandas as pd
import matplotlib.pyplot as plt
s = pd.Series([5, 10, 15, 20, 25, 30, 35])
rolling_avg = s.rolling(window=4).mean()
plt.plot(s, label='Original')
plt.plot(rolling_avg, label='Rolling Avg')
plt.legend()
plt.show()

A Scatter plot with original data points and rolling average points overlapping exactly

B Pie chart showing proportion of original data values

C Bar chart showing original data and rolling average side by side for each point

D Line plot with original data as a smooth increasing line and rolling average starting with NaN for first 3 points, then smooth line following original but lagging

💡 Hint

Rolling average smooths data and has NaN for initial points without enough data.

The rolling average line starts with NaN for the first 3 points (window=4), then follows the trend of original data but smoother and slightly lagged. A line plot best shows this.

🧠 Conceptual

advanced

Why use window functions instead of groupby aggregation?

Which statement best explains why window functions are useful compared to simple groupby aggregation?

A Window functions allow calculations across rows while keeping the original row structure, unlike groupby which reduces rows to aggregated results.

B Window functions always run faster than groupby aggregations.

C Groupby aggregation can only be used on numeric data, window functions work on all data types.

D Window functions replace the need for any filtering or sorting in data processing.

💡 Hint

Think about whether the number of rows changes after applying each method.

Window functions compute values across rows but keep the original number of rows, allowing more detailed analysis. Groupby aggregation reduces rows by summarizing groups.

🔧 Debug

expert

Identify the error in this expanding window code

What error does the following code produce and why? import pandas as pd s = pd.Series([2, 4, 6, 8]) result = s.expanding(min_periods=5).mean() print(result.tolist())

Pandas

import pandas as pd
s = pd.Series([2, 4, 6, 8])
result = s.expanding(min_periods=5).mean()
print(result.tolist())

A SyntaxError due to incorrect use of expanding function

B All values are NaN because min_periods=5 is greater than the series length

C TypeError because mean() cannot be applied to Series

D IndexError because expanding window exceeds series length

💡 Hint

Check the length of the series and the min_periods parameter.

The expanding window requires at least 5 data points to compute the mean, but the series has only 4 points, so all results are NaN.

Practice

1. What is the main benefit of using window functions in pandas?

easy

A. They allow calculations over a subset of rows without collapsing the data.

B. They delete rows that do not meet a condition.

C. They convert DataFrames into lists.

D. They only work on numeric columns.

Solution

1. Step 1: Understand window function purpose

   Window functions perform calculations across a set of rows related to the current row without reducing the number of rows.

2. Step 2: Compare options

   They allow calculations over a subset of rows without collapsing the data. correctly describes this benefit. Options A, B, and D describe unrelated or incorrect behaviors.

3. Final Answer:

   They allow calculations over a subset of rows without collapsing the data. -> Option A

4. Quick Check:

   Window functions keep all rows while calculating [OK]

Hint: Window functions keep all rows while calculating [OK]

Common Mistakes:

• Thinking window functions remove rows
• Confusing window functions with filtering
• Assuming window functions only work on numbers

2. Which of the following is the correct syntax to calculate a rolling mean over 3 rows in pandas?

easy

A. df.rolling['col'](3).mean()

B. df['col'].rolling(3).mean()

C. df['col'].mean(rolling=3)

D. df.rolling.mean(3)['col']

Solution

1. Step 1: Recall rolling mean syntax

   The correct syntax uses the rolling() method on the Series, then calls mean(): df['col'].rolling(3).mean()

2. Step 2: Check each option

   df['col'].rolling(3).mean() matches the correct syntax. Options B, C, and D misuse method calls or order.

3. Final Answer:

   df['col'].rolling(3).mean() -> Option B

4. Quick Check:

   rolling() then mean() is correct order [OK]

Hint: Use rolling() before mean() on the column [OK]

Common Mistakes:

• Using square brackets with rolling
• Calling mean() before rolling()
• Passing rolling window size inside mean()

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'value': [10, 20, 30, 40, 50]})
df['rolling_sum'] = df['value'].rolling(3).sum()
print(df)

What is the output of the rolling_sum column?

medium

A. [NaN, 30.0, 60.0, 90.0, 120.0]

B. [10, 30, 60, 90, 120]

C. [NaN, NaN, 60.0, 90.0, 120.0]

D. [10, 20, 30, 40, 50]

Solution

1. Step 1: Understand rolling sum with window=3

   Rolling sum with window 3 sums current and previous 2 rows. First two rows have less than 3 values, so result is NaN.

2. Step 2: Calculate sums

   Row 3: 10+20+30=60, Row 4: 20+30+40=90, Row 5: 30+40+50=120

3. Final Answer:

   [NaN, NaN, 60.0, 90.0, 120.0] -> Option C

4. Quick Check:

   Rolling sum skips incomplete windows [OK]

Hint: Rolling sums start producing values after full window [OK]

Common Mistakes:

• Expecting sums for first two rows
• Confusing rolling sum with cumulative sum
• Ignoring NaN for incomplete windows

4. Which of the following is true about this code snippet?

df['rank'] = df['score'].rolling(3).rank()

medium

A. The rolling window size must be larger than 3.

B. rank() requires a window size argument.

C. The code should use groupby before rolling.

D. rolling() supports rank() directly.

Solution

1. Step 1: Check rolling and rank compatibility

   The rolling() method returns a Rolling object that supports aggregation functions like sum(), mean(), rank(), etc.

2. Step 2: Identify correct statement

   rolling() supports rank() directly. The other options describe incorrect requirements or unrelated behaviors.

3. Final Answer:

   rolling() supports rank() directly. -> Option D

4. Quick Check:

   rank() is valid on rolling objects [OK]

Hint: rank() works directly on rolling objects [OK]

Common Mistakes:

• Thinking rank() is not supported directly on rolling
• Trying to pass window size to rank()
• Confusing rolling with groupby

5. You have daily sales data in a DataFrame and want to calculate a 7-day moving average of sales but keep all days in the result. Which pandas method should you use and why?

hard

A. Use rolling(7).mean() because it calculates moving averages without dropping rows.

B. Use groupby('day').mean() because it groups days and averages sales.

C. Use resample('7D').mean() because it resamples data into 7-day periods.

D. Use cumsum() because it gives cumulative sums over 7 days.

Solution

1. Step 1: Understand the goal

   You want a moving average over 7 days that keeps all original rows (days) in the DataFrame.

2. Step 2: Evaluate methods

   rolling(7).mean() calculates the moving average using a sliding window and keeps all rows. groupby() aggregates and reduces rows. resample() changes frequency and may drop days. cumsum() is cumulative sum, not average.

3. Final Answer:

   Use rolling(7).mean() because it calculates moving averages without dropping rows. -> Option A

4. Quick Check:

   rolling().mean() keeps all rows with moving average [OK]

Hint: rolling().mean() keeps all rows with moving average [OK]

Common Mistakes:

• Using groupby which reduces rows
• Using resample which changes data frequency
• Confusing cumulative sum with moving average