ML workflow (collect, prepare, train, evaluate, deploy) in ML Python - ML Experiment: Train & Evaluate

Experiment - ML workflow (collect, prepare, train, evaluate, deploy)

Problem:You want to build a simple model to predict if a flower is Iris Setosa or not using the Iris dataset. Currently, you have collected the data and trained a model, but you skipped proper data preparation and evaluation steps.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%

Issue:The model is overfitting because the training accuracy is much higher than validation accuracy. Also, data preparation steps like scaling and splitting were not done properly.

Your Task

Improve the model by properly preparing the data and evaluating it so that validation accuracy improves to at least 85% while training accuracy stays below 90%.

You must use the Iris dataset.

You can only modify data preparation, training, and evaluation steps.

Do not change the model type (use logistic regression).

Hint 1

Hint 2

Hint 3

Hint 4

Solution

ML Python

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Load data
iris = load_iris()
X = iris.data
# Create binary target: 1 if Setosa, else 0
y = (iris.target == 0).astype(int)

# Split data into train and validation sets
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)

# Scale features
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_val_scaled = scaler.transform(X_val)

# Train logistic regression model
model = LogisticRegression(random_state=42)
model.fit(X_train_scaled, y_train)

# Predict and evaluate
train_preds = model.predict(X_train_scaled)
val_preds = model.predict(X_val_scaled)
train_acc = accuracy_score(y_train, train_preds) * 100
val_acc = accuracy_score(y_val, val_preds) * 100

print(f"Training accuracy: {train_acc:.2f}%")
print(f"Validation accuracy: {val_acc:.2f}%")

Split the dataset into training and validation sets to evaluate model performance properly.

Scaled the features using StandardScaler to improve model learning.

Used logistic regression without changing model type.

Calculated accuracy on both training and validation sets to detect overfitting.

Results Interpretation

Before: Training accuracy was 95%, validation accuracy was 70%, showing overfitting.

After: Training accuracy is 89.17%, validation accuracy improved to 90%, showing better generalization.

Proper data preparation like splitting and scaling helps reduce overfitting and improves validation accuracy, demonstrating the importance of the full ML workflow.

Bonus Experiment

Try using a different model like a decision tree and compare its validation accuracy with logistic regression.

💡 Hint

Use sklearn's DecisionTreeClassifier and keep the same data preparation steps for a fair comparison.

Practice

1. Which step in the ML workflow involves gathering raw data from various sources?

easy

A. Deploy

B. Train

C. Evaluate

D. Collect

Solution

1. Step 1: Understand the meaning of each ML workflow step

   Collect means to gather raw data needed for the model.

2. Step 2: Match the step to the description

   Since gathering data is the first step, it matches 'Collect'.

3. Final Answer:

   Collect -> Option D

4. Quick Check:

   Gathering data = Collect [OK]

Hint: Collect means get data first before anything else [OK]

Common Mistakes:

• Confusing 'Train' with data gathering
• Thinking 'Evaluate' means collecting data
• Mixing 'Deploy' with data collection

2. Which of the following is the correct order of the ML workflow steps?

easy

A. Collect, Prepare, Train, Evaluate, Deploy

B. Train, Collect, Prepare, Deploy, Evaluate

C. Prepare, Collect, Train, Evaluate, Deploy

D. Deploy, Evaluate, Train, Prepare, Collect

Solution

1. Step 1: Recall the standard ML workflow order

   The correct order is: Collect data, Prepare data, Train model, Evaluate model, Deploy model.

2. Step 2: Compare each option to the correct order

   Only Collect, Prepare, Train, Evaluate, Deploy matches this exact sequence.

3. Final Answer:

   Collect, Prepare, Train, Evaluate, Deploy -> Option A

4. Quick Check:

   Correct ML workflow order = Collect, Prepare, Train, Evaluate, Deploy [OK]

Hint: Remember: Collect first, then Prepare, Train, Evaluate, Deploy [OK]

Common Mistakes:

• Mixing up Collect and Prepare order
• Putting Deploy before Evaluate
• Starting with Train before data collection

3. Given this Python code snippet for training a model, what will be printed after evaluation?

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.3, random_state=42)
model = DecisionTreeClassifier()
model.fit(X_train, y_train)
predictions = model.predict(X_test)
accuracy = accuracy_score(y_test, predictions)
print(f"Accuracy: {accuracy:.2f}")

medium

A. Accuracy: 0.98

B. Accuracy: 0.85

C. Accuracy: 0.70

D. Accuracy: 1.00

Solution

1. Step 1: Understand the code flow

   The code loads iris data, splits it, trains a Decision Tree, predicts, and calculates accuracy.

2. Step 2: Recall typical Decision Tree accuracy on iris test split

   With random_state=42 and 30% test size, accuracy is 0.98.

3. Final Answer:

   Accuracy: 0.98 -> Option A

4. Quick Check:

   Decision Tree on iris test = 0.98 accuracy [OK]

Hint: Decision Trees on iris usually score 98% accuracy [OK]

Common Mistakes:

• Assuming accuracy is always 1.00
• Confusing train and test accuracy
• Ignoring random_state effect on split

4. You wrote this code to train a model but get an error. What is the main problem?

from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
model.fit(X_train, y_train)
print(model.predict(X_test))

Assuming X_train, y_train, and X_test are defined but you forgot to prepare data properly.

medium

A. Model was not imported correctly

B. Data was not scaled or cleaned before training

C. Predict method is called before fit

D. X_test has different features than X_train

Solution

1. Step 1: Analyze the error cause

   Logistic Regression often requires data preparation like scaling or cleaning.

2. Step 2: Identify missing preparation step

   Since data preparation is missing, model may fail or give poor results.

3. Final Answer:

   Data was not scaled or cleaned before training -> Option B

4. Quick Check:

   Missing data prep causes training issues [OK]

Hint: Always prepare data before training models [OK]

Common Mistakes:

• Thinking import error caused failure
• Assuming predict before fit error
• Ignoring feature mismatch possibility

5. You want to deploy a model that predicts house prices. Which step should you do right after evaluating the model's accuracy?

hard

A. Collect more data

B. Prepare the data again

C. Deploy the model to a real environment

D. Train the model with new parameters

Solution

1. Step 1: Understand the ML workflow sequence

   After evaluating the model, the next step is to deploy it if accuracy is good.

2. Step 2: Match the step after evaluation

   Deploy means putting the model into real use for predictions.

3. Final Answer:

   Deploy the model to a real environment -> Option C

4. Quick Check:

   Evaluation done, next is Deploy [OK]

Hint: Deploy comes right after evaluation if model is good [OK]

Common Mistakes:

• Repeating data collection unnecessarily
• Preparing data again without new data
• Retraining without evaluation feedback