Compiling models (optimizer, loss, metrics) in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of model training metrics

Consider the following TensorFlow model compilation and training code snippet. What will be the printed output for the training accuracy after the first epoch?

TensorFlow

import tensorflow as tf
import numpy as np

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(2,), activation='sigmoid')
])

model.compile(optimizer='sgd', loss='binary_crossentropy', metrics=['accuracy'])

# Dummy data
x_train = np.array([[0,0],[0,1],[1,0],[1,1]])
y_train = np.array([0,1,1,0])

history = model.fit(x_train, y_train, epochs=1, verbose=0)
print(f"Training accuracy: {history.history['accuracy'][0]:.2f}")

A Training accuracy: 0.50

B Training accuracy: 1.00

C Training accuracy: 0.25

D Training accuracy: 0.00

💡 Hint

Think about the model's initial random weights and the simple dataset.

With random initial weights and only one epoch, the model is unlikely to perfectly classify all samples, so accuracy around 0.5 is expected.

❓ Model Choice

intermediate

Choosing the correct loss function for multi-class classification

You want to compile a TensorFlow model to classify images into 5 categories. The labels are one-hot encoded vectors. Which loss function should you choose?

A tf.keras.losses.SparseCategoricalCrossentropy()

B tf.keras.losses.CategoricalCrossentropy()

C tf.keras.losses.BinaryCrossentropy()

D tf.keras.losses.MeanSquaredError()

💡 Hint

One-hot encoded labels require a specific crossentropy loss.

CategoricalCrossentropy is used when labels are one-hot encoded. SparseCategoricalCrossentropy is for integer labels.

❓ Hyperparameter

advanced

Effect of optimizer choice on training speed

You compile two identical models with the same architecture and loss but different optimizers: Adam and SGD. Which statement about their training behavior is generally true?

A Adam adapts learning rates and often converges faster than SGD.

B SGD usually converges faster than Adam on most problems.

C Both optimizers always produce the same training speed and results.

D Adam requires manual learning rate decay to work properly.

💡 Hint

Think about how Adam adjusts learning rates automatically.

Adam optimizer adapts learning rates per parameter and often converges faster than vanilla SGD.

❓ Metrics

advanced

Interpreting multiple metrics in model.compile

If you compile a model with metrics=['accuracy', 'precision', 'recall'], what will be the output after training?

A The metrics will be ignored and only loss will be reported.

B Only 'accuracy' will be tracked because 'precision' and 'recall' are invalid metric names.

C The model will raise a ValueError due to invalid metric names.

D The training history will include keys 'accuracy', 'precision', and 'recall' with their values per epoch.

💡 Hint

TensorFlow supports standard metrics such as 'precision' and 'recall'.

TensorFlow recognizes 'precision' and 'recall' as valid metrics and tracks them along with accuracy.

🔧 Debug

expert

Identifying the cause of a metric reporting error

You compile a TensorFlow model with metrics=['accuracy'] but during training, you get this error: "ValueError: Shapes (None, 1) and (None, 10) are incompatible". What is the most likely cause?

A The loss function is missing from model.compile.

B The optimizer is incompatible with the accuracy metric.

C The model output shape does not match the shape of the labels provided.

D The batch size is too large for the model.

💡 Hint

Check the shapes of model outputs and labels carefully.

This error usually means the model predicts a shape different from the labels, causing metric calculation to fail.

Practice

1. What is the main purpose of the compile() method in a TensorFlow model?

easy

A. To set the optimizer, loss function, and metrics before training

B. To train the model on data

C. To save the model to disk

D. To make predictions on new data

Solution

1. Step 1: Understand the role of compile()

   The compile() method prepares the model for training by specifying how it learns and how performance is measured.

2. Step 2: Identify what compile() sets

   It sets the optimizer (how the model updates weights), the loss function (how error is calculated), and metrics (how performance is tracked).

3. Final Answer:

   To set the optimizer, loss function, and metrics before training -> Option A

4. Quick Check:

   Compile sets optimizer, loss, metrics = A [OK]

Hint: Compile sets learning rules and measurements before training [OK]

Common Mistakes:

• Confusing compile with training or prediction
• Thinking compile saves the model
• Assuming compile runs the training process

2. Which of the following is the correct way to compile a TensorFlow model with Adam optimizer, categorical crossentropy loss, and accuracy metric?

easy

A. model.compile(optimizer='adam', loss='mse', metrics='accuracy')

B. model.compile(optimizer='sgd', loss='mse', metrics=['accuracy'])

C. model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

D. model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

Solution

1. Step 1: Check optimizer and loss names

   The Adam optimizer is specified as 'adam' and categorical crossentropy loss as 'categorical_crossentropy'.

2. Step 2: Verify metrics format

   Metrics must be passed as a list, so ['accuracy'] is correct, not a string.

3. Final Answer:

   model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) -> Option C

4. Quick Check:

   Correct optimizer, loss, and metrics list = D [OK]

Hint: Use list for metrics and correct loss name [OK]

Common Mistakes:

• Passing metrics as a string instead of list
• Using wrong loss function for classification
• Choosing wrong optimizer name

3. Consider the code below:

model.compile(optimizer='sgd', loss='mse', metrics=['mae'])
history = model.fit(x_train, y_train, epochs=2)
print(history.history['mae'])

What will be printed?

medium

A. A single float value of mean absolute error after training

B. A list of mean squared error values for each epoch

C. An error because 'mae' is not a valid metric

D. A list of mean absolute error values for each epoch

Solution

1. Step 1: Understand metrics in compile and fit

   The model is compiled with 'mae' (mean absolute error) as a metric, so it will track this during training.

2. Step 2: Check what history.history['mae'] contains

   It stores a list of metric values for each epoch, so printing it shows a list of MAE values per epoch.

3. Final Answer:

   A list of mean absolute error values for each epoch -> Option D

4. Quick Check:

   Metrics list stores per-epoch values = B [OK]

Hint: history.history stores metric lists per epoch [OK]

Common Mistakes:

• Expecting a single float instead of list
• Confusing loss with metric values
• Thinking 'mae' is invalid metric

4. You wrote this code:

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics='accuracy')

What is the problem?

medium

A. Metrics should be a list, not a string

B. Loss function name is incorrect

C. Optimizer name is invalid

D. Model must be compiled after training

Solution

1. Step 1: Check metrics argument type

   Metrics must be passed as a list or tuple, e.g., ['accuracy'], not a string.

2. Step 2: Confirm other arguments are correct

   Optimizer 'adam' and loss 'categorical_crossentropy' are valid names, so the issue is due to metrics format.

3. Final Answer:

   Metrics should be a list, not a string -> Option A

4. Quick Check:

   Metrics argument must be list = A [OK]

Hint: Always pass metrics as a list, even if one metric [OK]

Common Mistakes:

• Passing metrics as a string
• Misnaming loss or optimizer
• Compiling after training instead of before

5. You want to compile a model for a binary classification task. Which combination of optimizer, loss, and metrics is the best choice?

hard

A. optimizer='rmsprop', loss='mse', metrics=['mae']

B. optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']

C. optimizer='sgd', loss='categorical_crossentropy', metrics=['accuracy']

D. optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']

Solution

1. Step 1: Identify task type

   Binary classification means two classes, so the loss should be 'binary_crossentropy'.

2. Step 2: Choose suitable optimizer and metrics

   Adam optimizer is widely used and effective; accuracy is a good metric for classification.

3. Step 3: Check other options

   Options B and D use categorical losses for multi-class, and A uses regression losses, so they are less suitable.

4. Final Answer:

   optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'] -> Option B

5. Quick Check:

   Binary task needs binary_crossentropy loss = C [OK]

Hint: Binary classification uses binary_crossentropy loss [OK]

Common Mistakes:

• Using categorical loss for binary tasks
• Choosing regression loss for classification
• Ignoring metric suitability