Simple neural network with scikit-learn in ML Python - Model Pipeline Trace

Model Pipeline - Simple neural network with scikit-learn

This pipeline shows how a simple neural network learns to classify data using scikit-learn. It starts with raw data, prepares it, trains the model, and then makes predictions.

Data Flow - 5 Stages

1Raw Data

150 rows x 4 columns→Load Iris dataset with 4 features per sample→150 rows x 4 columns

[5.1, 3.5, 1.4, 0.2]

↓

2Train/Test Split

150 rows x 4 columns→Split data into 120 training and 30 testing samples→120 rows x 4 columns (train), 30 rows x 4 columns (test)

[5.0, 3.6, 1.4, 0.2] (train), [6.7, 3.1, 4.7, 1.5] (test)

↓

3Feature Scaling

120 rows x 4 columns→Scale features to zero mean and unit variance→120 rows x 4 columns

[-0.9, 1.2, -1.3, -1.4]

↓

4Model Training

120 rows x 4 columns→Train MLPClassifier with 1 hidden layer of 5 neurons→Trained model

Model weights updated after each epoch

↓

5Prediction

30 rows x 4 columns→Model predicts class labels for test samples→30 rows x 1 column (predicted labels)

[0, 1, 2, 1, 0]

Training Trace - Epoch by Epoch

Loss
1.2 |****
0.9 |*** 
0.6 |**  
0.3 |*   
0.0 |    
     1 5 10 15 20 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.50	Model starts learning, loss is high, accuracy low
5	0.7	0.75	Loss decreases, accuracy improves
10	0.4	0.85	Model is learning well, loss continues to drop
15	0.25	0.90	Loss low, accuracy high, model converging
20	0.15	0.95	Training loss minimal, accuracy near perfect

Prediction Trace - 5 Layers

Layer 1: Input Layer

Layer 2: Feature Scaling

Layer 3: Hidden Layer (ReLU activation)

Layer 4: Output Layer (Softmax)

Layer 5: Prediction

Model Quiz - 3 Questions

What happens to the data shape after feature scaling?

A Number of columns reduces by half

B Number of rows doubles

C Shape stays the same but values change

D Data becomes one-dimensional

Key Insight

This visualization shows how a simple neural network learns by adjusting weights to reduce loss and improve accuracy. Feature scaling helps the model learn faster. The softmax layer turns outputs into probabilities for easy class prediction.

Practice

1. What is the main purpose of using MLPClassifier in scikit-learn?

easy

A. To perform data scaling and normalization

B. To create and train a simple neural network for classification tasks

C. To visualize data distributions

D. To split data into training and testing sets

Solution

1. Step 1: Understand the role of MLPClassifier

   MLPClassifier is a tool in scikit-learn designed to build and train neural networks for classification problems.

2. Step 2: Compare options with MLPClassifier's purpose

   The other options describe tasks like scaling, visualization, and data splitting, which are not the main function of MLPClassifier.

3. Final Answer:

   To create and train a simple neural network for classification tasks -> Option B

4. Quick Check:

   MLPClassifier = neural network classifier [OK]

Hint: MLPClassifier builds neural nets for classification [OK]

Common Mistakes:

• Confusing MLPClassifier with data preprocessing tools
• Thinking it is for regression only
• Assuming it splits data automatically

2. Which of the following is the correct way to import the MLPClassifier from scikit-learn?

easy

A. import neural_network.MLPClassifier from sklearn

B. import MLPClassifier from sklearn.network

C. from sklearn.ml import MLPClassifier

D. from sklearn.neural_network import MLPClassifier

Solution

1. Step 1: Recall the correct module path for MLPClassifier

   MLPClassifier is located in the sklearn.neural_network module.

2. Step 2: Check each import statement for syntax and correctness

   The other options have incorrect module names or import syntax.

3. Final Answer:

   from sklearn.neural_network import MLPClassifier -> Option D

4. Quick Check:

   Correct import = from sklearn.neural_network import MLPClassifier [OK]

Hint: MLPClassifier is in sklearn.neural_network module [OK]

Common Mistakes:

• Using wrong module names like sklearn.ml
• Incorrect import syntax
• Confusing neural_network with network

3. What will be the output of the following code snippet?

from sklearn.neural_network import MLPClassifier
X = [[0., 0.], [1., 1.]]
y = [0, 1]
clf = MLPClassifier(hidden_layer_sizes=(2,), max_iter=1000, random_state=1)
clf.fit(X, y)
pred = clf.predict([[2., 2.]])
print(pred)

medium

A. [0]

B. Error due to wrong input shape

C. [1]

D. [2]

Solution

1. Step 1: Understand the training data and labels

   The model is trained on two points: [0., 0.] labeled 0 and [1., 1.] labeled 1.

2. Step 2: Predict the label for input [2., 2.]

   Since [2., 2.] is closer to [1., 1.] which has label 1, the model predicts 1.

3. Final Answer:

   [1] -> Option C

4. Quick Check:

   Input near class 1 predicts 1 [OK]

Hint: Prediction matches closest trained label [OK]

Common Mistakes:

• Assuming output is 0 by default
• Confusing input shape errors
• Ignoring random_state effect

4. Identify the error in the following code snippet:

from sklearn.neural_network import MLPClassifier
X = [[0, 0], [1, 1]]
y = [0, 1]
clf = MLPClassifier(hidden_layer_sizes=2, max_iter=500)
clf.fit(X, y)

medium

A. hidden_layer_sizes should be a tuple, not an integer

B. max_iter value is too low

C. X and y have mismatched lengths

D. MLPClassifier requires data scaling before fitting

Solution

1. Step 1: Check the hidden_layer_sizes parameter

   hidden_layer_sizes expects a tuple indicating the number of neurons per hidden layer, e.g., (2,), not an integer.

2. Step 2: Verify other parameters and data

   max_iter=500 is valid, X and y lengths match, and scaling is recommended but not mandatory.

3. Final Answer:

   hidden_layer_sizes should be a tuple, not an integer -> Option A

4. Quick Check:

   hidden_layer_sizes = tuple [OK]

Hint: hidden_layer_sizes must be tuple like (2,) [OK]

Common Mistakes:

• Passing integer instead of tuple for hidden_layer_sizes
• Assuming max_iter causes error
• Thinking scaling is mandatory for fit

5. You want to improve your MLPClassifier model's accuracy on a complex dataset. Which of these changes is most likely to help?

hard

A. Increase the number of neurons in hidden layers and train longer

B. Reduce the number of hidden layers to zero

C. Remove the activation function to speed up training

D. Use fewer training samples to avoid overfitting

Solution

1. Step 1: Understand model complexity and training

   Increasing neurons and training longer allows the model to learn more complex patterns.

2. Step 2: Evaluate other options

   Reducing hidden layers to zero removes learning capacity, removing activation breaks non-linearity, and fewer samples usually reduce accuracy.

3. Final Answer:

   Increase the number of neurons in hidden layers and train longer -> Option A

4. Quick Check:

   More neurons + training = better learning [OK]

Hint: More neurons and training improve complex learning [OK]

Common Mistakes:

• Thinking fewer layers always improve accuracy
• Removing activation functions to simplify
• Using less data to prevent overfitting blindly