Compiling models (optimizer, loss, metrics) in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Compiling models (optimizer, loss, metrics)

Which metric matters for compiling models and WHY

When you compile a model, you choose an optimizer, a loss function, and metrics to watch. The loss shows how well the model is learning during training. The optimizer decides how the model changes to get better. The metrics help you understand the model's performance in ways that matter for your goal.

For example, if you want to classify images, accuracy might be a good metric. But if you want to detect rare events, precision or recall might be better. Choosing the right loss and metrics helps you know if your model is improving and if it will work well in real life.

Confusion matrix example

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |    TP=50 |   FN=10
      Negative           |    FP=5  |   TN=35
    

This matrix helps calculate metrics like precision and recall, which you can add as metrics when compiling your model.

Precision vs Recall tradeoff with examples

Precision tells you how many predicted positives are actually correct. High precision means fewer false alarms.

Recall tells you how many actual positives you found. High recall means you miss fewer real cases.

Example: For a spam filter, high precision is important so you don't mark good emails as spam. For a cancer detector, high recall is important so you don't miss any cancer cases.

Good vs Bad metric values when compiling models

Good: Loss decreases steadily during training, and metrics like accuracy or F1 score improve. For example, accuracy rising from 60% to 90% means the model is learning well.

Bad: Loss stays high or jumps around, and metrics do not improve or get worse. For example, accuracy stuck at 50% (random guessing) means the model is not learning.

Common pitfalls when compiling models

• Choosing the wrong loss function for your task (e.g., using regression loss for classification) can stop learning.
• Using metrics that don't match your goal (e.g., accuracy for imbalanced data) can mislead you.
• Ignoring overfitting signs: training loss goes down but validation loss goes up.
• Data leakage: metrics look great but model sees test data during training.

Self-check question

Your model has 98% accuracy but 12% recall on fraud detection. Is it good for production? Why or why not?

Answer: No, it is not good. The model misses 88% of fraud cases (low recall), which is dangerous. Even with high accuracy, the model fails to catch most frauds, so you should improve recall.

Key Result

Choosing the right loss and metrics during model compilation ensures meaningful training and evaluation aligned with your task goals.

Practice

1. What is the main purpose of the compile() method in a TensorFlow model?

easy

A. To set the optimizer, loss function, and metrics before training

B. To train the model on data

C. To save the model to disk

D. To make predictions on new data

Solution

1. Step 1: Understand the role of compile()

   The compile() method prepares the model for training by specifying how it learns and how performance is measured.

2. Step 2: Identify what compile() sets

   It sets the optimizer (how the model updates weights), the loss function (how error is calculated), and metrics (how performance is tracked).

3. Final Answer:

   To set the optimizer, loss function, and metrics before training -> Option A

4. Quick Check:

   Compile sets optimizer, loss, metrics = A [OK]

Hint: Compile sets learning rules and measurements before training [OK]

Common Mistakes:

• Confusing compile with training or prediction
• Thinking compile saves the model
• Assuming compile runs the training process

2. Which of the following is the correct way to compile a TensorFlow model with Adam optimizer, categorical crossentropy loss, and accuracy metric?

easy

A. model.compile(optimizer='adam', loss='mse', metrics='accuracy')

B. model.compile(optimizer='sgd', loss='mse', metrics=['accuracy'])

C. model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

D. model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

Solution

1. Step 1: Check optimizer and loss names

   The Adam optimizer is specified as 'adam' and categorical crossentropy loss as 'categorical_crossentropy'.

2. Step 2: Verify metrics format

   Metrics must be passed as a list, so ['accuracy'] is correct, not a string.

3. Final Answer:

   model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) -> Option C

4. Quick Check:

   Correct optimizer, loss, and metrics list = D [OK]

Hint: Use list for metrics and correct loss name [OK]

Common Mistakes:

• Passing metrics as a string instead of list
• Using wrong loss function for classification
• Choosing wrong optimizer name

3. Consider the code below:

model.compile(optimizer='sgd', loss='mse', metrics=['mae'])
history = model.fit(x_train, y_train, epochs=2)
print(history.history['mae'])

What will be printed?

medium

A. A single float value of mean absolute error after training

B. A list of mean squared error values for each epoch

C. An error because 'mae' is not a valid metric

D. A list of mean absolute error values for each epoch

Solution

1. Step 1: Understand metrics in compile and fit

   The model is compiled with 'mae' (mean absolute error) as a metric, so it will track this during training.

2. Step 2: Check what history.history['mae'] contains

   It stores a list of metric values for each epoch, so printing it shows a list of MAE values per epoch.

3. Final Answer:

   A list of mean absolute error values for each epoch -> Option D

4. Quick Check:

   Metrics list stores per-epoch values = B [OK]

Hint: history.history stores metric lists per epoch [OK]

Common Mistakes:

• Expecting a single float instead of list
• Confusing loss with metric values
• Thinking 'mae' is invalid metric

4. You wrote this code:

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics='accuracy')

What is the problem?

medium

A. Metrics should be a list, not a string

B. Loss function name is incorrect

C. Optimizer name is invalid

D. Model must be compiled after training

Solution

1. Step 1: Check metrics argument type

   Metrics must be passed as a list or tuple, e.g., ['accuracy'], not a string.

2. Step 2: Confirm other arguments are correct

   Optimizer 'adam' and loss 'categorical_crossentropy' are valid names, so the issue is due to metrics format.

3. Final Answer:

   Metrics should be a list, not a string -> Option A

4. Quick Check:

   Metrics argument must be list = A [OK]

Hint: Always pass metrics as a list, even if one metric [OK]

Common Mistakes:

• Passing metrics as a string
• Misnaming loss or optimizer
• Compiling after training instead of before

5. You want to compile a model for a binary classification task. Which combination of optimizer, loss, and metrics is the best choice?

hard

A. optimizer='rmsprop', loss='mse', metrics=['mae']

B. optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']

C. optimizer='sgd', loss='categorical_crossentropy', metrics=['accuracy']

D. optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']

Solution

1. Step 1: Identify task type

   Binary classification means two classes, so the loss should be 'binary_crossentropy'.

2. Step 2: Choose suitable optimizer and metrics

   Adam optimizer is widely used and effective; accuracy is a good metric for classification.

3. Step 3: Check other options

   Options B and D use categorical losses for multi-class, and A uses regression losses, so they are less suitable.

4. Final Answer:

   optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'] -> Option B

5. Quick Check:

   Binary task needs binary_crossentropy loss = C [OK]

Hint: Binary classification uses binary_crossentropy loss [OK]

Common Mistakes:

• Using categorical loss for binary tasks
• Choosing regression loss for classification
• Ignoring metric suitability