Moving averages in ML Python - Model Pipeline Trace

Model Pipeline - Moving averages

This pipeline uses moving averages to smooth noisy data. It helps the model see clearer trends by averaging recent points before training.

Data Flow - 4 Stages

1Raw data input

1000 rows x 1 column→Collect raw time series data points→1000 rows x 1 column

[23, 25, 22, 24, 26, 28, 27, 29, 30, 31, ...]

↓

2Calculate moving average

1000 rows x 1 column→Compute 5-point moving average to smooth data→996 rows x 1 column

[24.0, 25.0, 25.4, 26.8, 28.0, 29.0, 29.8, 31.0, 32.0, ...]

↓

3Train/test split

996 rows x 1 column→Split data into 80% training and 20% testing sets→Train: 796 rows x 1 column, Test: 200 rows x 1 column

Train: [24.0, 25.0, ..., 29.0], Test: [29.8, 31.0, ...]

↓

4Model training

Train: 796 rows x 1 column→Train simple regression model on smoothed data→Trained model

Model learns trend from moving average values

Training Trace - Epoch by Epoch

Loss
0.15 |*****
0.10 |****
0.07 |***
0.05 |**
0.04 |*
      +---------
       1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.15	0.60	Model starts learning, loss is high, accuracy low
2	0.10	0.72	Loss decreases, accuracy improves
3	0.07	0.80	Model fits data better, smoother predictions
4	0.05	0.85	Loss continues to drop, accuracy rises
5	0.04	0.88	Training converges, good fit on smoothed data

Prediction Trace - 3 Layers

Layer 1: Input raw data sample

Layer 2: Calculate 5-point moving average

Layer 3: Model prediction

Model Quiz - 3 Questions

What does the moving average step do to the data?

A Randomly changes data values

B Removes all data points

C Smooths the data by averaging recent points

D Splits data into training and testing sets

Key Insight

Using moving averages helps the model see clearer trends by reducing noise. This smoothing improves training stability and prediction accuracy.

Practice

1. What is the main purpose of using a moving average in data analysis?

easy

A. To smooth out short-term fluctuations and highlight longer-term trends

B. To increase the number of data points in a dataset

C. To remove all noise from the data completely

D. To predict exact future values without error

Solution

1. Step 1: Understand the role of moving averages

   Moving averages smooth data by averaging nearby points, reducing short-term ups and downs.

2. Step 2: Identify the main goal

   The goal is to reveal longer-term trends by reducing noise, not to remove noise completely or predict exact values.

3. Final Answer:

   To smooth out short-term fluctuations and highlight longer-term trends -> Option A

4. Quick Check:

   Moving average = smoothing trends [OK]

Hint: Moving averages smooth data to show trends clearly [OK]

Common Mistakes:

• Thinking moving averages increase data points
• Believing moving averages remove all noise
• Assuming moving averages predict exact future values

2. Which of the following Python code snippets correctly computes a simple moving average with window size 3 for a list data?

easy

A. [data[i] / 3 for i in range(len(data))]

B. [sum(data[i:i+3]) for i in range(len(data)-3)]

C. [sum(data[i:i+3]) / 3 for i in range(len(data)-3)]

D. [(data[i] + data[i+1] + data[i+2]) / 3 for i in range(len(data)-2)]

Solution

1. Step 1: Understand moving average calculation

   A simple moving average with window 3 averages each group of 3 consecutive elements.

2. Step 2: Check each option's correctness

   [(data[i] + data[i+1] + data[i+2]) / 3 for i in range(len(data)-2)] correctly sums three consecutive elements and divides by 3, iterating till len(data)-2.
   [sum(data[i:i+3]) for i in range(len(data)-3)] sums but does not divide by 3.
   [sum(data[i:i+3]) / 3 for i in range(len(data)-3)] divides but uses range(len(data)-3), which is too short.
   [data[i] / 3 for i in range(len(data))] divides single elements by 3, not averaging groups.

3. Final Answer:

   [(data[i] + data[i+1] + data[i+2]) / 3 for i in range(len(data)-2)] -> Option D

4. Quick Check:

   Sum 3 elements / 3, range correct = [(data[i] + data[i+1] + data[i+2]) / 3 for i in range(len(data)-2)] [OK]

Hint: Sum 3 elements and divide by 3, loop till len-2 [OK]

Common Mistakes:

• Forgetting to divide by window size
• Using wrong range length causing index errors
• Averaging single elements instead of groups

3. Given the code below, what is the output?

data = [2, 4, 6, 8, 10]
window = 2
moving_avg = [sum(data[i:i+window]) / window for i in range(len(data) - window + 1)]
print(moving_avg)

medium

A. [2.0, 4.0, 6.0, 8.0, 10.0]

B. [3.0, 5.0, 7.0]

C. [3.0, 5.0, 7.0, 9.0]

D. [6.0, 8.0, 10.0]

Solution

1. Step 1: Calculate moving averages manually

   Window size is 2, so average pairs:
   (2+4)/2=3.0
   (4+6)/2=5.0
   (6+8)/2=7.0
   (8+10)/2=9.0

2. Step 2: Confirm output list length and values

   Length is len(data)-window+1 = 5-2+1=4, matching 4 values above.

3. Final Answer:

   [3.0, 5.0, 7.0, 9.0] -> Option C

4. Quick Check:

   Pairs averaged = [3.0, 5.0, 7.0, 9.0] [OK]

Hint: Average pairs sliding by one, length = len - window + 1 [OK]

Common Mistakes:

• Confusing window size with output length
• Calculating sums but forgetting to divide
• Off-by-one errors in range length

4. The following code is intended to compute a moving average with window size 3, but it misses the last window. What is the problem?

data = [1, 2, 3, 4, 5]
window = 3
moving_avg = [sum(data[i:i+window]) / window for i in range(len(data)-window)]
print(moving_avg)

medium

A. The range should be len(data) - window + 1 to include the last window

B. The window size is too large for the data list

C. sum() cannot be used on list slices

D. Division by window size should be outside the list comprehension

Solution

1. Step 1: Analyze the range length

   Range is len(data)-window = 5-3=2, but to cover all windows it should be len(data)-window+1 = 3.

2. Step 2: Understand impact of incorrect range

   Using len(data)-window misses the last valid window slice, causing incomplete results.

3. Final Answer:

   The range should be len(data) - window + 1 to include the last window -> Option A

4. Quick Check:

   Range length = len - window + 1 [OK]

Hint: Use range(len(data) - window + 1) for full coverage [OK]

Common Mistakes:

• Using len(data) - window instead of +1
• Thinking sum() can't handle slices
• Misplacing division outside comprehension

5. You have daily sales data for 10 days: [10, 12, 11, 14, 13, 15, 16, 14, 13, 12]. You want to smooth this data using a moving average with window size 4 but only want to keep averages where the window's average is greater than 13. Which Python code correctly computes this filtered moving average?

hard

A. [sum(data[i:i+4])/4 for i in range(len(data)-4) if sum(data[i:i+4])/4 > 13]

B. [avg for i in range(len(data)-3) if (avg := sum(data[i:i+4])/4) > 13]

C. [sum(data[i:i+4])/4 for i in range(len(data)-3) if sum(data[i:i+4]) > 13]

D. [sum(data[i:i+4])/4 for i in range(len(data)-3) if sum(data[i:i+4])/4 < 13]

Solution

1. Step 1: Understand window size and range

   Window size 4 means averaging groups of 4 elements, so range is len(data)-3 = 10-3=7.

2. Step 2: Filter averages greater than 13

   [avg for i in range(len(data)-3) if (avg := sum(data[i:i+4])/4) > 13] uses assignment expression to compute average once and filter if > 13.
   [sum(data[i:i+4])/4 for i in range(len(data)-4) if sum(data[i:i+4])/4 > 13] uses wrong range (len(data)-4=6), missing last window.
   [sum(data[i:i+4])/4 for i in range(len(data)-3) if sum(data[i:i+4]) > 13] filters sum > 13, not average > 13.
   [sum(data[i:i+4])/4 for i in range(len(data)-3) if sum(data[i:i+4])/4 < 13] filters averages less than 13, opposite condition.

3. Final Answer:

   [avg for i in range(len(data)-3) if (avg := sum(data[i:i+4])/4) > 13] -> Option B

4. Quick Check:

   Use assignment expression to filter averages > 13 [OK]

Hint: Use assignment expression (walrus) to filter averages [OK]

Common Mistakes:

• Using wrong range length missing last windows
• Filtering sum instead of average
• Using wrong comparison operator