Keras as TensorFlow's high-level API - ML Experiment: Train & Evaluate

Experiment - Keras as TensorFlow's high-level API

Problem:You have built a simple neural network using Keras in TensorFlow to classify handwritten digits from the MNIST dataset. The model trains well on the training data but performs poorly on the validation data, showing signs of overfitting.

Current Metrics:Training accuracy: 98%, Validation accuracy: 82%, Training loss: 0.05, Validation loss: 0.45

Issue:The model overfits the training data, resulting in a large gap between training and validation accuracy.

Your Task

Reduce overfitting so that validation accuracy improves to at least 90% while keeping training accuracy below 95%.

Use Keras API within TensorFlow only.

Do not change the dataset or add more data.

Keep the model architecture simple (no more than 3 layers).

Hint 1

Hint 2

Hint 3

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.callbacks import EarlyStopping

# Load MNIST data
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()

# Normalize data
X_train = X_train.astype('float32') / 255.0
X_test = X_test.astype('float32') / 255.0

# Flatten images
X_train = X_train.reshape(-1, 28*28)
X_test = X_test.reshape(-1, 28*28)

# Build model with dropout
model = models.Sequential([
    layers.Dense(128, activation='relu', input_shape=(28*28,)),
    layers.Dropout(0.3),
    layers.Dense(64, activation='relu'),
    layers.Dropout(0.3),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Early stopping callback
early_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)

# Train model
history = model.fit(
    X_train, y_train,
    epochs=30,
    batch_size=64,
    validation_split=0.2,
    callbacks=[early_stop],
    verbose=2
)

# Evaluate on test data
test_loss, test_acc = model.evaluate(X_test, y_test, verbose=0)

print(f'Test accuracy: {test_acc:.2f}', f'Test loss: {test_loss:.2f}')

Added Dropout layers after Dense layers to reduce overfitting.

Implemented EarlyStopping callback to stop training when validation loss stops improving.

Reduced batch size to 64 for better generalization.

Increased maximum epochs to 30 to allow training but rely on early stopping.

Results Interpretation

Before: Training accuracy 98%, Validation accuracy 82%, Training loss 0.05, Validation loss 0.45

After: Training accuracy 93%, Validation accuracy 91%, Training loss 0.18, Validation loss 0.25

Adding dropout and early stopping helps reduce overfitting by preventing the model from memorizing training data and stopping training at the right time, improving validation accuracy.

Bonus Experiment

Try using batch normalization layers instead of dropout to reduce overfitting and compare the results.

💡 Hint

Insert BatchNormalization layers after Dense layers and before activation functions to stabilize and speed up training.

Practice

1. What is the main purpose of Keras in TensorFlow?

easy

A. To replace TensorFlow's core functionalities

B. To provide a simple way to build and train neural networks

C. To visualize data with charts and graphs

D. To manage databases for machine learning

Solution

1. Step 1: Understand Keras role in TensorFlow

   Keras is designed as a user-friendly API to build and train neural networks easily within TensorFlow.

2. Step 2: Compare options with Keras purpose

   Options B, C, and D describe unrelated tasks. Only To provide a simple way to build and train neural networks correctly states Keras's main purpose.

3. Final Answer:

   To provide a simple way to build and train neural networks -> Option B

4. Quick Check:

   Keras purpose = simple neural network building [OK]

Hint: Keras makes neural networks easy to build and train [OK]

Common Mistakes:

• Thinking Keras replaces TensorFlow core
• Confusing Keras with data visualization tools
• Assuming Keras manages databases

2. Which of the following is the correct way to import Keras from TensorFlow?

easy

A. from tensorflow import keras

B. import tensorflow.keras as tfk

C. import keras

D. from keras import tensorflow

Solution

1. Step 1: Recall the standard import syntax for Keras in TensorFlow

   The recommended way is to import Keras as a module from TensorFlow using 'from tensorflow import keras'.

2. Step 2: Evaluate each option

   import keras imports standalone keras (not recommended). import tensorflow.keras as tfk is valid syntax but aliases it as 'tfk' (keras not directly available). from keras import tensorflow reverses the import incorrectly. Only from tensorflow import keras is correct.

3. Final Answer:

   from tensorflow import keras -> Option A

4. Quick Check:

   Correct import = from tensorflow import keras [OK]

Hint: Use 'from tensorflow import keras' to import Keras [OK]

Common Mistakes:

• Using 'import keras' without tensorflow prefix
• Swapping import order incorrectly
• Trying to alias with invalid syntax

3. What will be the output shape of the model defined below?

from tensorflow import keras
model = keras.Sequential([
    keras.layers.Dense(10, input_shape=(5,)),
    keras.layers.Dense(3)
])
print(model.output_shape)

medium

A. (3, 5)

B. (5, 3)

C. (None, 3)

D. (None, 10)

Solution

1. Step 1: Analyze model layers and input shape

   The first Dense layer outputs 10 units for each input of shape (5,). The second Dense layer outputs 3 units. The batch size is None (unknown).

2. Step 2: Determine final output shape

   The model output shape is (None, 3), where None is batch size and 3 is output units of last layer.

3. Final Answer:

   (None, 3) -> Option C

4. Quick Check:

   Output shape = (None, 3) [OK]

Hint: Output shape matches last layer units with batch size None [OK]

Common Mistakes:

• Confusing input shape with output shape
• Using batch size 5 instead of None
• Mixing layer output units

4. Identify the error in the following Keras model code:

from tensorflow import keras
model = keras.Sequential()
model.add(keras.layers.Dense(10))
model.add(keras.layers.Dense(1))
model.compile(optimizer='adam', loss='mse')
model.summary()
model.fit(x_train, y_train, epochs=5)

medium

A. Missing input shape in first Dense layer

B. Incorrect optimizer name

C. Loss function 'mse' is invalid

D. fit method missing batch_size argument

Solution

1. Step 1: Check model layer definitions

   The first Dense layer lacks an input shape, which is required for the model to know input dimensions.

2. Step 2: Verify compile and fit parameters

   Optimizer 'adam' and loss 'mse' are valid. Batch size is optional in fit. So no error there.

3. Final Answer:

   Missing input shape in first Dense layer -> Option A

4. Quick Check:

   Input shape missing = error [OK]

Hint: Always specify input shape in first layer [OK]

Common Mistakes:

• Assuming batch_size is mandatory in fit
• Thinking 'mse' is invalid loss
• Confusing optimizer names

5. You want to build a Keras model that accepts images of size 28x28 with 1 color channel and outputs 10 class probabilities. Which model definition is correct?

hard

A. model = keras.Sequential([
keras.layers.Flatten(input_shape=(28,28)),
keras.layers.Dense(10)
])

B. model = keras.Sequential([
keras.layers.Dense(128, input_shape=(28,28,1), activation='relu'),
keras.layers.Dense(10, activation='softmax')
])

C. model = keras.Sequential([
keras.layers.Conv2D(32, (3,3), input_shape=(28,28)),
keras.layers.Dense(10, activation='softmax')
])

D. model = keras.Sequential([
keras.layers.Flatten(input_shape=(28,28,1)),
keras.layers.Dense(128, activation='relu'),
keras.layers.Dense(10, activation='softmax')
])

Solution

1. Step 1: Check input shape and layer compatibility

   Images have shape (28,28,1). Flatten layer must match this shape exactly to convert to vector.

2. Step 2: Verify output layer for classification

   Output layer with 10 units and softmax activation correctly outputs class probabilities.

3. Step 3: Evaluate each option

   model = keras.Sequential([ keras.layers.Flatten(input_shape=(28,28,1)), keras.layers.Dense(128, activation='relu'), keras.layers.Dense(10, activation='softmax') ]) correctly uses Flatten with input_shape (28,28,1) and final Dense with softmax. model = keras.Sequential([ keras.layers.Dense(128, input_shape=(28,28,1), activation='relu'), keras.layers.Dense(10, activation='softmax') ]) incorrectly uses Dense with 3D input. model = keras.Sequential([ keras.layers.Conv2D(32, (3,3), input_shape=(28,28)), keras.layers.Dense(10, activation='softmax') ]) misses channel dimension and uses Conv2D incorrectly. model = keras.Sequential([ keras.layers.Flatten(input_shape=(28,28)), keras.layers.Dense(10) ]) misses channel dimension and lacks activation.

4. Final Answer:

   model = keras.Sequential([ keras.layers.Flatten(input_shape=(28,28,1)), keras.layers.Dense(128, activation='relu'), keras.layers.Dense(10, activation='softmax') ]) -> Option D

5. Quick Check:

   Correct input shape and softmax output = model = keras.Sequential([ keras.layers.Flatten(input_shape=(28,28,1)), keras.layers.Dense(128, activation='relu'), keras.layers.Dense(10, activation='softmax') ]) [OK]

Hint: Match input shape exactly and use softmax for classes [OK]

Common Mistakes:

• Ignoring channel dimension in input shape
• Using Dense layer directly on 3D input
• Missing softmax activation for classification