Sequential model API in TensorFlow - Model Pipeline Trace

Model Pipeline - Sequential model API

The Sequential model API in TensorFlow helps us build a simple stack of layers for machine learning. We add layers one after another, and the model learns to make predictions from input data.

Data Flow - 5 Stages

1Input Data

1000 rows x 20 columns→Raw numerical features representing samples→1000 rows x 20 columns

[[0.5, 1.2, ..., 0.3], [0.1, 0.4, ..., 0.9], ...]

↓

2Normalization Layer

1000 rows x 20 columns→Scale features to have mean 0 and variance 1→1000 rows x 20 columns

[[-0.1, 0.5, ..., -0.3], [0.0, -0.2, ..., 1.1], ...]

↓

3Dense Layer 1

1000 rows x 20 columns→Fully connected layer with 64 neurons and ReLU activation→1000 rows x 64 columns

[[0.0, 1.2, ..., 0.5], [0.3, 0.7, ..., 0.9], ...]

↓

4Dense Layer 2

1000 rows x 64 columns→Fully connected layer with 32 neurons and ReLU activation→1000 rows x 32 columns

[[0.2, 0.1, ..., 0.4], [0.5, 0.3, ..., 0.7], ...]

↓

5Output Layer

1000 rows x 32 columns→Fully connected layer with 1 neuron and sigmoid activation for binary classification→1000 rows x 1 column

[[0.8], [0.3], [0.6], ...]

Training Trace - Epoch by Epoch

Loss
0.7 |****
0.6 |*** 
0.5 |**  
0.4 |*   
0.3 |*   
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.60	Model starts learning, loss is high, accuracy is low
2	0.50	0.75	Loss decreases, accuracy improves
3	0.40	0.82	Model continues to learn well
4	0.35	0.86	Loss decreases steadily, accuracy increases
5	0.30	0.89	Good convergence, model is improving

Prediction Trace - 5 Layers

Layer 1: Input Layer

Layer 2: Normalization Layer

Layer 3: Dense Layer 1 (ReLU)

Layer 4: Dense Layer 2 (ReLU)

Layer 5: Output Layer (Sigmoid)

Model Quiz - 3 Questions

What does the ReLU activation function do in the Dense layers?

A It outputs values between -1 and 1

B It scales all values between 0 and 1

C It changes all negative values to zero and keeps positive values as is

D It sums all inputs without change

Key Insight

The Sequential model API lets us build a simple chain of layers easily. Each layer transforms data step-by-step, and training improves the model by reducing loss and increasing accuracy. Activation functions like ReLU help the model learn complex patterns by introducing non-linearity.

Practice

1. What is the main purpose of the Sequential model API in TensorFlow?

easy

A. To visualize the training process of a model

B. To create complex models with multiple inputs and outputs

C. To perform data preprocessing before training

D. To build a model by stacking layers in a linear order

Solution

1. Step 1: Understand the Sequential API purpose

   The Sequential API is designed to build models by stacking layers one after another in a simple linear fashion.

2. Step 2: Compare options with the API's function

   Options B, C, and D describe other functionalities not related to the Sequential API's main purpose.

3. Final Answer:

   To build a model by stacking layers in a linear order -> Option D

4. Quick Check:

   Sequential API = linear stacking of layers [OK]

Hint: Sequential means layers stacked one after another [OK]

Common Mistakes:

• Confusing Sequential with Functional API for complex models
• Thinking Sequential handles data preprocessing
• Assuming Sequential is for visualization

2. Which of the following is the correct way to create a Sequential model with one dense layer of 10 units in TensorFlow?

easy

A. model = Sequential(Dense(10))

B. model = Sequential([Dense(10)])

C. model = Sequential().add(Dense(10))

D. model = Sequential.add(Dense(10))

Solution

1. Step 1: Recall correct Sequential model creation syntax

   The Sequential model can be created by passing a list of layers inside the constructor, e.g., Sequential([Dense(10)]).

2. Step 2: Check each option's syntax validity

   model = Sequential([Dense(10)]) uses the correct list syntax. model = Sequential(Dense(10)) misses the list brackets. model = Sequential().add(Dense(10)) is valid usage but requires assignment to a variable to keep the model reference. model = Sequential.add(Dense(10)) incorrectly calls add() on the class, not an instance.

3. Final Answer:

   model = Sequential([Dense(10)]) -> Option B

4. Quick Check:

   Sequential needs list of layers in constructor [OK]

Hint: Pass layers as a list inside Sequential() [OK]

Common Mistakes:

• Omitting brackets around layers list
• Calling add() on class instead of instance
• Chaining add() without assignment

3. What will be the output shape of the model after running this code?

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

model = Sequential([
    Dense(5, input_shape=(10,)),
    Dense(3)
])
print(model.output_shape)

medium

A. (None, 5)

B. (10, 3)

C. (None, 3)

D. (5, 3)

Solution

1. Step 1: Understand input and output shapes in Sequential

   The input shape is (10,), so the first Dense layer outputs (None, 5). The second Dense layer outputs (None, 3) because it has 3 units.

2. Step 2: Identify final output shape

   The model's output shape is the output of the last layer, which is (None, 3). None means batch size is flexible.

3. Final Answer:

   (None, 3) -> Option C

4. Quick Check:

   Last Dense units = output shape [OK]

Hint: Output shape matches last layer units with batch None [OK]

Common Mistakes:

• Confusing input shape with output shape
• Using batch size instead of None
• Mixing layer output shapes

4. Identify the error in this Sequential model code:

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

model = Sequential()
model.add(Dense(10, input_shape=(5,)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(x_train, y_train, epochs=5)

medium

A. x_train and y_train are not defined

B. Sequential model cannot use add() method

C. Loss function 'mse' is invalid

D. Missing import for optimizer

Solution

1. Step 1: Check code for missing definitions

   The code uses x_train and y_train in model.fit() but they are not defined anywhere, causing a runtime error.

2. Step 2: Verify other parts

   Optimizer 'adam' and loss 'mse' are valid strings. add() method is valid for Sequential instances. Imports are sufficient.

3. Final Answer:

   x_train and y_train are not defined -> Option A

4. Quick Check:

   Undefined training data causes error [OK]

Hint: Check if training data variables are defined before fit() [OK]

Common Mistakes:

• Assuming loss 'mse' is invalid
• Thinking add() method is not allowed
• Ignoring missing data variables

5. You want to build a Sequential model for a classification task with 3 classes. Which of the following is the best final layer and loss combination?

hard

A. Dense(3, activation='softmax') with loss='categorical_crossentropy'

B. Dense(1, activation='sigmoid') with loss='mean_squared_error'

C. Dense(3, activation='relu') with loss='binary_crossentropy'

D. Dense(3) with loss='sparse_categorical_crossentropy'

Solution

1. Step 1: Understand classification output requirements

   For 3 classes, the final layer should have 3 units with softmax activation to output class probabilities.

2. Step 2: Match appropriate loss function

   For one-hot encoded labels, 'categorical_crossentropy' is the correct loss function to use with softmax output.

3. Final Answer:

   Dense(3, activation='softmax') with loss='categorical_crossentropy' -> Option A

4. Quick Check:

   Softmax + categorical_crossentropy = multi-class classification [OK]

Hint: Use softmax + categorical_crossentropy for multi-class tasks [OK]

Common Mistakes:

• Using sigmoid for multi-class output
• Using mean squared error for classification
• Missing activation in final layer