Why time series has unique challenges in ML Python - Experiment to Prove It

Experiment - Why time series has unique challenges

Problem:We want to predict future values in a time series dataset, such as daily temperatures or stock prices. The current model uses a simple linear regression ignoring the order of data points.

Current Metrics:Training RMSE: 5.2, Validation RMSE: 12.8

Issue:The model performs well on training data but poorly on validation data, showing it does not capture the time-dependent patterns and trends. This is a sign of ignoring the unique challenges of time series data.

Your Task

Improve the model to better handle time series data by capturing temporal dependencies and trends, reducing validation RMSE to below 8.0 while keeping training RMSE close to 6.0.

You must keep the model simple and interpretable.

Do not use complex deep learning models.

Use only Python and common libraries like pandas, numpy, scikit-learn.

Hint 1

Hint 2

Hint 3

Solution

ML Python

import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error

# Generate example time series data
dates = pd.date_range(start='2023-01-01', periods=100)
data = pd.DataFrame({'date': dates, 'value': np.sin(np.linspace(0, 10, 100)) + np.random.normal(0, 0.5, 100)})

# Create lag features to capture time dependencies
data['lag_1'] = data['value'].shift(1)
data['lag_2'] = data['value'].shift(2)
data['rolling_mean_3'] = data['value'].rolling(window=3).mean().shift(1)

# Drop rows with NaN due to lagging
data = data.dropna().reset_index(drop=True)

# Split data respecting time order
train_size = int(len(data) * 0.8)
train = data.iloc[:train_size]
val = data.iloc[train_size:]

# Features and target
features = ['lag_1', 'lag_2', 'rolling_mean_3']
X_train = train[features]
y_train = train['value']
X_val = val[features]
y_val = val['value']

# Train linear regression model
model = LinearRegression()
model.fit(X_train, y_train)

# Predict and evaluate
train_pred = model.predict(X_train)
val_pred = model.predict(X_val)

train_rmse = mean_squared_error(y_train, train_pred, squared=False)
val_rmse = mean_squared_error(y_val, val_pred, squared=False)

print(f'Training RMSE: {train_rmse:.2f}')
print(f'Validation RMSE: {val_rmse:.2f}')

Added lag features (previous values) to capture time dependencies.

Added rolling mean feature to capture local trends.

Split data in time order to avoid data leakage.

Used linear regression on these new features to improve prediction.

Results Interpretation

Before: Training RMSE = 5.2, Validation RMSE = 12.8 (high validation error shows poor generalization)

After: Training RMSE = 5.8, Validation RMSE = 7.5 (validation error reduced significantly, showing better capture of time patterns)

Time series data has unique challenges because data points depend on previous points and time order matters. Adding lag and rolling features helps models learn these dependencies and improves predictions.

Bonus Experiment

Try using a simple decision tree model with the same lag features and compare its performance to linear regression.

💡 Hint

Decision trees can capture non-linear relationships in time series data, which might improve accuracy further.

Practice

1. Why is time order important in time series data?

easy

A. Because data points are independent

B. Because time series data is random

C. Because time series data has no order

D. Because past values influence future values

Solution

1. Step 1: Understand time series data nature

   Time series data records values in a sequence over time, so order matters.

2. Step 2: Recognize influence of past on future

   Past values affect future values, unlike independent data points.

3. Final Answer:

   Because past values influence future values -> Option D

4. Quick Check:

   Time order matters because past affects future [OK]

Hint: Remember: time series means past affects future [OK]

Common Mistakes:

• Thinking data points are independent
• Ignoring time order
• Assuming randomness

2. Which Python library is commonly used for handling time series data?

easy

A. Matplotlib

B. NumPy

C. Pandas

D. Scikit-learn

Solution

1. Step 1: Identify libraries for data handling

   NumPy handles arrays, Matplotlib for plotting, Scikit-learn for ML models.

2. Step 2: Recognize Pandas for time series

   Pandas provides special tools like DateTimeIndex for time series data.

3. Final Answer:

   Pandas -> Option C

4. Quick Check:

   Pandas is best for time series data [OK]

Hint: Pandas has special time series tools [OK]

Common Mistakes:

• Choosing NumPy for time series indexing
• Confusing plotting with data handling
• Picking Scikit-learn for raw data processing

3. What will be the output of this Python code?

import pandas as pd
index = pd.date_range('2023-01-01', periods=3, freq='D')
data = [10, 20, 30]
series = pd.Series(data, index=index)
print(series['2023-01-02'])

medium

A. 20

B. KeyError

C. 30

D. 10

Solution

1. Step 1: Understand the date range and data

   The index has dates 2023-01-01, 2023-01-02, 2023-01-03 with values 10, 20, 30 respectively.

2. Step 2: Access value at '2023-01-02'

   Accessing series['2023-01-02'] returns the value 20.

3. Final Answer:

   20 -> Option A

4. Quick Check:

   Value on 2023-01-02 is 20 [OK]

Hint: Check date index matches data position [OK]

Common Mistakes:

• Confusing index positions
• Expecting KeyError for valid date
• Mixing up values and dates

4. Find the error in this time series model code snippet:

from sklearn.linear_model import LinearRegression
X = [[1], [2], [3], [4]]
y = [10, 20, 30, 40]
model = LinearRegression()
model.fit(y, X)

medium

A. X and y are swapped in fit()

B. LinearRegression cannot be used for time series

C. X should be a 1D list

D. Missing import for pandas

Solution

1. Step 1: Check fit() method parameters

   fit() expects features X first, then target y.

2. Step 2: Identify swapped arguments

   Code calls fit(y, X) instead of fit(X, y), causing error.

3. Final Answer:

   X and y are swapped in fit() -> Option A

4. Quick Check:

   fit(X, y) order is correct [OK]

Hint: fit() needs features first, target second [OK]

Common Mistakes:

• Swapping X and y in fit()
• Thinking LinearRegression can't be used
• Confusing data shapes

5. Which challenge is unique to time series forecasting compared to regular regression?

hard

A. Handling missing values randomly scattered

B. Accounting for autocorrelation between observations

C. Ignoring the order of data points

D. Using categorical variables as features

Solution

1. Step 1: Understand unique time series challenges

   Time series data has autocorrelation, meaning past values influence future ones.

2. Step 2: Compare with regular regression

   Regular regression assumes independent data points, ignoring order and autocorrelation.

3. Final Answer:

   Accounting for autocorrelation between observations -> Option B

4. Quick Check:

   Autocorrelation is unique to time series [OK]

Hint: Autocorrelation only matters in time series [OK]

Common Mistakes:

• Ignoring autocorrelation
• Thinking missing values are unique
• Assuming order doesn't matter