Window functions (expanding, ewm) in Data Analysis Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is an expanding window function in data analysis?

An expanding window function calculates a statistic over all data points from the start up to the current point, growing the window as it moves forward.

intermediate

Explain the difference between an expanding window and a rolling window.

An expanding window grows from the start to the current point, including all previous data, while a rolling window uses a fixed-size window that moves along the data.

beginner

What does the ewm (exponentially weighted moving) function do?

The ewm function calculates statistics giving more weight to recent data points, making the result more sensitive to recent changes.

intermediate

How does the 'span' parameter affect the ewm function?

The 'span' controls how quickly the weights decrease for older data; a smaller span means more weight on recent points, while a larger span smooths more over time.

beginner

Show a simple Python example using pandas to calculate an expanding mean.

import pandas as pd
s = pd.Series([1, 2, 3, 4, 5])
expanding_mean = s.expanding().mean()
print(expanding_mean)

This prints the mean of all values from the start up to each point.

What does the expanding() function in pandas do?

A Calculates statistics over all data from the start to current point

B Calculates statistics over a fixed-size window

C Calculates statistics giving more weight to recent data

D Calculates statistics only on the last data point

The expanding() function grows the window from the start to the current point, including all previous data.

Which window function gives more importance to recent data points?

A expanding()

B rolling()

C cumsum()

D ewm()

The ewm() function applies exponentially decreasing weights to older data, emphasizing recent points.

In ewm(span=5), what does the span control?

A The total number of data points used

B The size of the fixed window

C How fast weights decrease for older data

D The minimum number of observations

Span controls the decay rate of weights in ewm; smaller span means faster decay.

Which pandas function would you use to calculate a rolling average over the last 3 data points?

A rolling(window=3)

B ewm(span=3)

C expanding()

D cumsum()

rolling(window=3) calculates statistics over a fixed window of 3 data points.

What will s.expanding().mean() return for s = pd.Series([2, 4, 6])?

A [2, 4, 6]

B [2, 3, 4]

C [2, 3, 5]

D [2, 4, 5]

The expanding mean is calculated as [2, (2+4)/2=3, (2+4+6)/3=4].

Describe how expanding window functions work and give a real-life example where they might be useful.

Think about tracking a running average that includes all past data.

Explain the purpose of the ewm function and how changing its parameters affects the result.

Consider how recent events might be more important in stock prices.

Practice

1. What does the expanding() window function do in data analysis?

easy

A. Calculates statistics using a fixed-size sliding window

B. Calculates statistics using only the last 3 data points

C. Calculates statistics by giving more weight to recent data points

D. Calculates statistics using all data points up to the current one

Solution

1. Step 1: Understand expanding() function

   The expanding() function includes all data points from the start up to the current point for calculations.

2. Step 2: Compare with other window types

   Unlike rolling windows or ewm, expanding uses all past data, not a fixed window or weighted recent data.

3. Final Answer:

   Calculates statistics using all data points up to the current one -> Option D

4. Quick Check:

   expanding() = all data so far [OK]

Hint: expanding() uses all data up to now, no fixed window [OK]

Common Mistakes:

• Confusing expanding() with rolling() fixed window
• Thinking ewm() is the same as expanding()
• Assuming expanding() only uses recent data

2. Which of the following is the correct syntax to calculate the exponentially weighted mean of a pandas Series data with a span of 3?

easy

A. data.ewm(span=3).mean()

B. data.expanding(span=3).mean()

C. data.rolling(window=3).ewm().mean()

D. data.ewm(window=3).mean()

Solution

1. Step 1: Recall ewm() syntax

   The ewm() function uses parameters like span to control decay; span=3 is valid.

2. Step 2: Check each option

   data.ewm(span=3).mean() uses correct syntax: data.ewm(span=3).mean(). Others misuse parameters or combine methods incorrectly.

3. Final Answer:

   data.ewm(span=3).mean() -> Option A

4. Quick Check:

   ewm(span=3) is correct syntax [OK]

Hint: Use ewm(span=3).mean() for weighted mean with span 3 [OK]

Common Mistakes:

• Using expanding() instead of ewm() for weighted mean
• Passing window parameter to ewm() which is invalid
• Chaining rolling() before ewm() incorrectly

3. Given the pandas Series data = pd.Series([10, 20, 30, 40]), what is the output of data.expanding().mean().tolist()?

medium

A. [10.0, 15.0, 20.0, 25.0]

B. [10.0, 20.0, 30.0, 40.0]

C. [10.0, 15.0, 25.0, 35.0]

D. [10.0, 20.0, 25.0, 30.0]

Solution

1. Step 1: Calculate expanding mean step-by-step

   Expanding mean at each point is average of all values up to that index:
   Index 0: 10 / 1 = 10.0
   Index 1: (10+20)/2 = 15.0
   Index 2: (10+20+30)/3 = 20.0
   Index 3: (10+20+30+40)/4 = 25.0

2. Step 2: Match calculated list with options

   The list is [10.0, 15.0, 20.0, 25.0], which matches [10.0, 15.0, 20.0, 25.0] exactly.

3. Final Answer:

   [10.0, 15.0, 20.0, 25.0] -> Option A

4. Quick Check:

   Expanding mean averages all so far [OK]

Hint: Expanding mean averages all data up to current index [OK]

Common Mistakes:

• Calculating rolling mean instead of expanding mean
• Using last value only instead of average
• Miscomputing cumulative sums

4. You run the code data.ewm(span=2).mean() but get an error. Which fix will solve the problem?

medium

A. Import pandas as pd before using data

B. Replace span with window parameter

C. Use data.ewm(span=2).mean() on a pandas Series, not a list

D. Call mean() before ewm()

Solution

1. Step 1: Identify cause of error

   The ewm() method is a pandas Series method. If data is a plain list, it causes an error.

2. Step 2: Correct usage context

   Convert data to a pandas Series before calling ewm(), e.g., pd.Series(data).ewm(span=2).mean().

3. Final Answer:

   Use data.ewm(span=2).mean() on a pandas Series, not a list -> Option C

4. Quick Check:

   ewm() needs pandas Series [OK]

Hint: ewm() works only on pandas Series, not plain lists [OK]

Common Mistakes:

• Trying to use ewm() on Python lists
• Using wrong parameter names like window in ewm()
• Calling mean() before ewm()

5. You have daily sales data with some missing days. You want to smooth the sales trend giving more weight to recent days but still include all past data. Which pandas window function and parameter combination is best?

hard

A. data.expanding(min_periods=5).mean() to average all data after 5 days

B. data.ewm(span=5, adjust=False).mean() to weight recent days more

C. data.rolling(window=5).mean() to use last 5 days only

D. data.expanding().mean() to average all data equally

Solution

1. Step 1: Understand smoothing with weights

   To give more weight to recent days, ewm() is appropriate because it applies exponential decay.

2. Step 2: Consider missing days and all past data

   ewm() includes all past data but weights recent points more; adjust=False gives a smoother result.

3. Step 3: Compare options

   data.ewm(span=5, adjust=False).mean() to weight recent days more uses ewm(span=5, adjust=False) which fits the requirement best. Others either use equal weights or fixed windows.

4. Final Answer:

   data.ewm(span=5, adjust=False).mean() -> Option B

5. Quick Check:

   ewm() weights recent data more [OK]

Hint: Use ewm(span=5, adjust=False) for weighted smoothing with all past data [OK]

Common Mistakes:

• Using expanding() which weights all data equally
• Using rolling() which ignores older data beyond window
• Not setting adjust=False for smoother ewm