Linear regression concept in ML Python - ML Experiment: Train & Evaluate

Experiment - Linear regression concept

Problem:You want to predict house prices based on their size using a simple linear regression model.

Current Metrics:Training loss: 1200, Validation loss: 1500

Issue:The model is underfitting, shown by high loss on both training and validation sets.

Your Task

Improve the linear regression model to reduce training and validation loss below 500.

Use only linear regression without adding polynomial features.

Do not change the dataset.

Hint 1

Hint 2

Hint 3

Solution

ML Python

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_squared_error

# Sample data: house sizes (in square feet) and prices (in $1000s)
X = np.array([[500], [750], [1000], [1250], [1500], [1750], [2000], [2250], [2500], [2750]])
y = np.array([150, 200, 250, 300, 350, 400, 450, 500, 550, 600])

# Split data into training and validation sets
X_train, X_val = X[:8], X[8:]
y_train, y_val = y[:8], y[8:]

# Scale features
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_val_scaled = scaler.transform(X_val)

# Create and train linear regression model
model = LinearRegression()
model.fit(X_train_scaled, y_train)

# Predict and calculate loss
train_preds = model.predict(X_train_scaled)
val_preds = model.predict(X_val_scaled)

train_loss = mean_squared_error(y_train, train_preds)
val_loss = mean_squared_error(y_val, val_preds)

print(f"Training loss: {train_loss:.2f}")
print(f"Validation loss: {val_loss:.2f}")

Scaled the input feature using StandardScaler to normalize data.

Kept the linear regression model but improved input data quality.

Used mean squared error to measure loss for better clarity.

Results Interpretation

Before scaling: Training loss = 1200, Validation loss = 1500

After scaling: Training loss = 0.00, Validation loss = 0.00

Scaling input features helps linear regression learn better by making data easier to interpret, reducing underfitting and improving prediction accuracy.

Bonus Experiment

Try adding polynomial features (like square of house size) to see if the model fits better.

💡 Hint

Use sklearn's PolynomialFeatures to create new features and then fit the linear regression model again.

Practice

1. What does linear regression primarily do in machine learning?

easy

A. It groups data points into clusters.

B. It reduces the number of features in data.

C. It finds a straight line to predict values based on input data.

D. It creates decision trees for classification.

Solution

1. Step 1: Understand the goal of linear regression

   Linear regression aims to find a straight line that best fits the data points to predict continuous values.

2. Step 2: Compare with other methods

   Clustering groups data, feature reduction simplifies data, and decision trees classify data, which are different tasks.

3. Final Answer:

   It finds a straight line to predict values based on input data. -> Option C

4. Quick Check:

   Linear regression = straight line prediction [OK]

Hint: Linear regression fits a line to predict numbers [OK]

Common Mistakes:

• Confusing regression with classification
• Thinking it groups data instead of predicting
• Mixing it up with feature reduction techniques

2. Which of the following is the correct formula for a simple linear regression model?

easy

A. y = m - x / b

B. y = mx + b

C. y = m + x * b

D. y = x / m + b

Solution

1. Step 1: Recall the linear regression formula

   The formula is y = mx + b, where m is slope and b is intercept.

2. Step 2: Check each option

   Only y = mx + b matches the correct formula exactly.

3. Final Answer:

   y = mx + b -> Option B

4. Quick Check:

   Linear formula = y = mx + b [OK]

Hint: Slope times x plus intercept equals y [OK]

Common Mistakes:

• Mixing multiplication and division signs
• Swapping slope and intercept positions
• Using incorrect operators between terms

3. Given the linear regression model y = 2x + 3, what is the predicted value when x = 4?

medium

A. 11

B. 14

C. 8

D. 10

Solution

1. Step 1: Substitute x = 4 into the model

   Calculate y = 2 * 4 + 3.

2. Step 2: Perform the calculation

   2 * 4 = 8, then 8 + 3 = 11.

3. Final Answer:

   11 -> Option A

4. Quick Check:

   2*4+3 = 11 [OK]

Hint: Multiply slope by x, then add intercept [OK]

Common Mistakes:

• Adding before multiplying
• Using wrong slope or intercept values
• Confusing x with y

4. Identify the error in this Python code for linear regression prediction:

def predict(x):
    return 2 * x +

print(predict(5))

medium

A. Syntax error due to incomplete return statement

B. Missing intercept value in the formula

C. Wrong function name for prediction

D. Incorrect multiplication operator

Solution

1. Step 1: Check the return statement syntax

   The return line ends with a plus sign and no value after it, causing a syntax error.

2. Step 2: Confirm error type

   Python requires complete expressions; incomplete return causes syntax error.

3. Final Answer:

   Syntax error due to incomplete return statement -> Option A

4. Quick Check:

   Incomplete return line = syntax error [OK]

Hint: Check if return line ends abruptly [OK]

Common Mistakes:

• Ignoring incomplete expression errors
• Assuming missing intercept is the error
• Thinking function name matters here

5. You have data points: (1,3), (2,5), (3,7). Which linear regression model best fits this data?

hard

A. y = x + 2

B. y = 3x + 0

C. y = 2x + 3

D. y = 2x + 1

Solution

1. Step 1: Calculate slope (m)

   Change in y: 5-3=2, 7-5=2; change in x: 2-1=1, 3-2=1; slope m = 2/1 = 2.

2. Step 2: Calculate intercept (b)

   Use point (1,3): 3 = 2*1 + b -> b = 3 - 2 = 1.

3. Final Answer:

   y = 2x + 1 -> Option D

4. Quick Check:

   Slope=2, intercept=1 fits points [OK]

Hint: Slope = change in y/change in x, intercept from point [OK]

Common Mistakes:

• Mixing slope and intercept values
• Using wrong points for calculation
• Assuming intercept is zero without checking