Keras as TensorFlow's high-level API - Model Metrics & Evaluation

Metrics & Evaluation - Keras as TensorFlow's high-level API

Which metric matters for this concept and WHY

Keras is used to build and train models easily. The key metrics depend on the task: for classification, accuracy, precision, recall, and F1 score matter. For regression, mean squared error or mean absolute error matter. These metrics tell us how well the model learned using Keras.

Confusion matrix or equivalent visualization (ASCII)

For classification tasks, Keras models often use a confusion matrix to show results:

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |    TP    |   FN
      Negative           |    FP    |   TN
    

This helps calculate precision, recall, and accuracy from Keras model predictions.

Precision vs Recall tradeoff with concrete examples

When using Keras for classification, precision and recall trade off:

• High precision: Few false alarms. Good for spam filters so real emails aren't marked spam.
• High recall: Few missed positives. Good for medical tests so sick patients aren't missed.

Keras lets you tune models to balance these by changing thresholds or loss functions.

What "good" vs "bad" metric values look like for this use case

Using Keras, a good classification model might have:

• Accuracy above 85%
• Precision and recall above 80%
• F1 score close to precision and recall

Bad models have low accuracy (near random), or very unbalanced precision and recall (e.g., 95% precision but 10% recall).

Metrics pitfalls (accuracy paradox, data leakage, overfitting indicators)

• Accuracy paradox: High accuracy can be misleading if classes are imbalanced.
• Data leakage: If test data leaks into training, metrics look falsely good.
• Overfitting: High training accuracy but low test accuracy means model memorized data, not learned.
• Ignoring recall or precision: Only looking at accuracy can hide poor performance on important classes.

Self-check question

Your Keras model has 98% accuracy but 12% recall on fraud cases. Is it good for production? Why not?

Answer: No, it is not good. The model misses 88% of fraud cases (low recall), which is dangerous. High accuracy is misleading because fraud is rare. You need better recall to catch fraud.

Key Result

Keras model metrics like precision, recall, and accuracy must be balanced and interpreted carefully to ensure real-world usefulness.

Practice

1. What is the main purpose of Keras in TensorFlow?

easy

A. To replace TensorFlow's core functionalities

B. To provide a simple way to build and train neural networks

C. To visualize data with charts and graphs

D. To manage databases for machine learning

Solution

1. Step 1: Understand Keras role in TensorFlow

   Keras is designed as a user-friendly API to build and train neural networks easily within TensorFlow.

2. Step 2: Compare options with Keras purpose

   Options B, C, and D describe unrelated tasks. Only To provide a simple way to build and train neural networks correctly states Keras's main purpose.

3. Final Answer:

   To provide a simple way to build and train neural networks -> Option B

4. Quick Check:

   Keras purpose = simple neural network building [OK]

Hint: Keras makes neural networks easy to build and train [OK]

Common Mistakes:

• Thinking Keras replaces TensorFlow core
• Confusing Keras with data visualization tools
• Assuming Keras manages databases

2. Which of the following is the correct way to import Keras from TensorFlow?

easy

A. from tensorflow import keras

B. import tensorflow.keras as tfk

C. import keras

D. from keras import tensorflow

Solution

1. Step 1: Recall the standard import syntax for Keras in TensorFlow

   The recommended way is to import Keras as a module from TensorFlow using 'from tensorflow import keras'.

2. Step 2: Evaluate each option

   import keras imports standalone keras (not recommended). import tensorflow.keras as tfk is valid syntax but aliases it as 'tfk' (keras not directly available). from keras import tensorflow reverses the import incorrectly. Only from tensorflow import keras is correct.

3. Final Answer:

   from tensorflow import keras -> Option A

4. Quick Check:

   Correct import = from tensorflow import keras [OK]

Hint: Use 'from tensorflow import keras' to import Keras [OK]

Common Mistakes:

• Using 'import keras' without tensorflow prefix
• Swapping import order incorrectly
• Trying to alias with invalid syntax

3. What will be the output shape of the model defined below?

from tensorflow import keras
model = keras.Sequential([
    keras.layers.Dense(10, input_shape=(5,)),
    keras.layers.Dense(3)
])
print(model.output_shape)

medium

A. (3, 5)

B. (5, 3)

C. (None, 3)

D. (None, 10)

Solution

1. Step 1: Analyze model layers and input shape

   The first Dense layer outputs 10 units for each input of shape (5,). The second Dense layer outputs 3 units. The batch size is None (unknown).

2. Step 2: Determine final output shape

   The model output shape is (None, 3), where None is batch size and 3 is output units of last layer.

3. Final Answer:

   (None, 3) -> Option C

4. Quick Check:

   Output shape = (None, 3) [OK]

Hint: Output shape matches last layer units with batch size None [OK]

Common Mistakes:

• Confusing input shape with output shape
• Using batch size 5 instead of None
• Mixing layer output units

4. Identify the error in the following Keras model code:

from tensorflow import keras
model = keras.Sequential()
model.add(keras.layers.Dense(10))
model.add(keras.layers.Dense(1))
model.compile(optimizer='adam', loss='mse')
model.summary()
model.fit(x_train, y_train, epochs=5)

medium

A. Missing input shape in first Dense layer

B. Incorrect optimizer name

C. Loss function 'mse' is invalid

D. fit method missing batch_size argument

Solution

1. Step 1: Check model layer definitions

   The first Dense layer lacks an input shape, which is required for the model to know input dimensions.

2. Step 2: Verify compile and fit parameters

   Optimizer 'adam' and loss 'mse' are valid. Batch size is optional in fit. So no error there.

3. Final Answer:

   Missing input shape in first Dense layer -> Option A

4. Quick Check:

   Input shape missing = error [OK]

Hint: Always specify input shape in first layer [OK]

Common Mistakes:

• Assuming batch_size is mandatory in fit
• Thinking 'mse' is invalid loss
• Confusing optimizer names

5. You want to build a Keras model that accepts images of size 28x28 with 1 color channel and outputs 10 class probabilities. Which model definition is correct?

hard

A. model = keras.Sequential([
keras.layers.Flatten(input_shape=(28,28)),
keras.layers.Dense(10)
])

B. model = keras.Sequential([
keras.layers.Dense(128, input_shape=(28,28,1), activation='relu'),
keras.layers.Dense(10, activation='softmax')
])

C. model = keras.Sequential([
keras.layers.Conv2D(32, (3,3), input_shape=(28,28)),
keras.layers.Dense(10, activation='softmax')
])

D. model = keras.Sequential([
keras.layers.Flatten(input_shape=(28,28,1)),
keras.layers.Dense(128, activation='relu'),
keras.layers.Dense(10, activation='softmax')
])

Solution

1. Step 1: Check input shape and layer compatibility

   Images have shape (28,28,1). Flatten layer must match this shape exactly to convert to vector.

2. Step 2: Verify output layer for classification

   Output layer with 10 units and softmax activation correctly outputs class probabilities.

3. Step 3: Evaluate each option

   model = keras.Sequential([ keras.layers.Flatten(input_shape=(28,28,1)), keras.layers.Dense(128, activation='relu'), keras.layers.Dense(10, activation='softmax') ]) correctly uses Flatten with input_shape (28,28,1) and final Dense with softmax. model = keras.Sequential([ keras.layers.Dense(128, input_shape=(28,28,1), activation='relu'), keras.layers.Dense(10, activation='softmax') ]) incorrectly uses Dense with 3D input. model = keras.Sequential([ keras.layers.Conv2D(32, (3,3), input_shape=(28,28)), keras.layers.Dense(10, activation='softmax') ]) misses channel dimension and uses Conv2D incorrectly. model = keras.Sequential([ keras.layers.Flatten(input_shape=(28,28)), keras.layers.Dense(10) ]) misses channel dimension and lacks activation.

4. Final Answer:

   model = keras.Sequential([ keras.layers.Flatten(input_shape=(28,28,1)), keras.layers.Dense(128, activation='relu'), keras.layers.Dense(10, activation='softmax') ]) -> Option D

5. Quick Check:

   Correct input shape and softmax output = model = keras.Sequential([ keras.layers.Flatten(input_shape=(28,28,1)), keras.layers.Dense(128, activation='relu'), keras.layers.Dense(10, activation='softmax') ]) [OK]

Hint: Match input shape exactly and use softmax for classes [OK]

Common Mistakes:

• Ignoring channel dimension in input shape
• Using Dense layer directly on 3D input
• Missing softmax activation for classification