Training data preparation in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Training data preparation

Problem:You want to train a text generation AI model, but your training data is messy. It has duplicate sentences, inconsistent formatting, and some irrelevant content.

Current Metrics:Training loss: 0.15, Validation loss: 0.45, Validation accuracy: 60%

Issue:The model is overfitting and not generalizing well because the training data quality is poor and inconsistent.

Your Task

Clean and prepare the training data to reduce overfitting and improve validation accuracy to at least 75%.

You cannot change the model architecture or training parameters.

You must only modify the training data preparation steps.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Prompt Engineering / GenAI

import re
from sklearn.model_selection import train_test_split

# Sample raw data
raw_data = [
    "Hello world!  ",
    "Hello world!",
    "This is a test.",
    "Irrelevant content here.",
    "Another sentence.",
    "Another sentence.",
    "  This is a test.  "
]

# Step 1: Remove duplicates and strip spaces
cleaned_data = list(set(sentence.strip().lower() for sentence in raw_data))

# Step 2: Filter out irrelevant content (e.g., sentences containing 'irrelevant')
filtered_data = [s for s in cleaned_data if 'irrelevant' not in s]

# Step 3: Split into training and validation sets
train_data, val_data = train_test_split(filtered_data, test_size=0.3, random_state=42)

# Show prepared data
print(f"Training data: {train_data}")
print(f"Validation data: {val_data}")

# Note: This prepared data would then be used for model training.

Removed duplicate sentences to reduce bias.

Converted all text to lowercase and stripped extra spaces for consistency.

Filtered out irrelevant sentences to improve data quality.

Split data into training and validation sets properly.

Results Interpretation

Before: Training loss: 0.15, Validation loss: 0.45, Validation accuracy: 60%

After: Training loss: 0.18, Validation loss: 0.30, Validation accuracy: 78%

Cleaning and preparing training data properly helps the model learn better patterns and generalize well, reducing overfitting and improving validation accuracy.

Bonus Experiment

Try augmenting the training data by adding synonyms or paraphrased sentences to increase data diversity.

💡 Hint

Use simple text augmentation techniques like replacing words with synonyms or rephrasing sentences to create more varied training examples.

Practice

1. What is the main purpose of training data preparation in machine learning?

easy

A. To clean and organize data for better model learning

B. To create the final model architecture

C. To deploy the model to production

D. To write the code for model training

Solution

1. Step 1: Understand the role of training data preparation

   Training data preparation involves cleaning and organizing data so the model can learn effectively.

2. Step 2: Differentiate from other steps in machine learning

   Creating model architecture, deployment, and coding are separate steps after data preparation.

3. Final Answer:

   To clean and organize data for better model learning -> Option A

4. Quick Check:

   Training data preparation = cleaning and organizing data [OK]

Hint: Focus on data cleaning and organizing for training [OK]

Common Mistakes:

• Confusing data preparation with model building
• Thinking deployment is part of data preparation
• Assuming coding is data preparation

2. Which of the following is the correct way to split data into training and testing sets in Python using scikit-learn?

easy

A. split_train_test(data, 0.2)

B. train_test(data, split=0.2)

C. train_test_split(data, test_size=0.2)

D. test_train_split(data, size=0.2)

Solution

1. Step 1: Recall the scikit-learn function for splitting data

   The correct function is train_test_split with parameters like test_size.

2. Step 2: Check the syntax of each option

   Only train_test_split(data, test_size=0.2) uses the correct function name and parameter syntax.

3. Final Answer:

   train_test_split(data, test_size=0.2) -> Option C

4. Quick Check:

   Correct function and parameter = train_test_split(data, test_size=0.2) [OK]

Hint: Remember scikit-learn's train_test_split function name [OK]

Common Mistakes:

• Using wrong function names
• Incorrect parameter names
• Mixing order of parameters

3. Given the code below, what will be the output of print(X_train.shape, X_test.shape)?

from sklearn.model_selection import train_test_split
import numpy as np
X = np.arange(20).reshape(10, 2)
X_train, X_test = train_test_split(X, test_size=0.3, random_state=42)

medium

A. (7, 2) (3, 2)

B. (3, 2) (7, 2)

C. (10, 2) (0, 2)

D. (5, 2) (5, 2)

Solution

1. Step 1: Understand the data shape and split ratio

   The data X has 10 rows and 2 columns. test_size=0.3 means 30% data for testing (3 rows) and 70% for training (7 rows).

2. Step 2: Calculate the shapes of training and testing sets

   Training set shape: (7, 2), Testing set shape: (3, 2).

3. Final Answer:

   (7, 2) (3, 2) -> Option A

4. Quick Check:

   70% train = 7 rows, 30% test = 3 rows [OK]

Hint: Calculate rows by multiplying total by split ratio [OK]

Common Mistakes:

• Swapping train and test sizes
• Ignoring the shape's second dimension
• Misunderstanding test_size meaning

4. Identify the error in the following code snippet for normalizing data using MinMaxScaler:

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
X = [[-1, 2], [-0.5, 6], [0, 10], [1, 18]]
X_scaled = scaler.fit_transform(X)
print(X_scaled)

medium

A. MinMaxScaler cannot handle negative values

B. MinMaxScaler requires data as a numpy array, not list

C. fit_transform should be called on scaler.fit(X).transform(X)

D. No error, code runs correctly

Solution

1. Step 1: Check input data type compatibility

   MinMaxScaler accepts lists or numpy arrays as input, so list input is valid.

2. Step 2: Verify method usage

   Calling scaler.fit_transform(X) is the correct way to fit and transform data in one step.

3. Final Answer:

   No error, code runs correctly -> Option D

4. Quick Check:

   MinMaxScaler works with lists and fit_transform method [OK]

Hint: MinMaxScaler accepts lists and arrays directly [OK]

Common Mistakes:

• Thinking input must be numpy array
• Misusing fit and transform methods
• Assuming scaler rejects negative values

5. You have a dataset with categorical text features and numeric features. Which sequence of steps correctly prepares the data for training a machine learning model?

hard

A. Split data, encode categorical features, normalize numeric features, then clean missing values

B. Clean missing values, encode categorical features, normalize numeric features, then split data

C. Normalize numeric features, clean missing values, split data, then encode categorical features

D. Encode categorical features, split data, clean missing values, then normalize numeric features

Solution

1. Step 1: Clean missing values first

   Cleaning missing data ensures no errors during encoding or normalization.

2. Step 2: Encode categorical features before normalization

   Categorical data must be converted to numbers before normalization.

3. Step 3: Normalize numeric features and then split data

   Normalization scales numeric data; splitting last avoids data leakage.

4. Final Answer:

   Clean missing values, encode categorical features, normalize numeric features, then split data -> Option B

5. Quick Check:

   Proper order: clean -> encode -> normalize -> split [OK]

Hint: Always clean first, encode before normalize, split last [OK]

Common Mistakes:

• Splitting data before cleaning causes leakage
• Normalizing before encoding categorical data
• Encoding after splitting leads to inconsistent categories