L1 and L2 regularization in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Difference between L1 and L2 regularization

Which statement correctly describes the main difference between L1 and L2 regularization in machine learning models?

A L2 regularization produces sparse models by setting weights to zero, while L1 regularization only shrinks weights without making them zero.

B L1 regularization tends to produce sparse models by driving some weights exactly to zero, while L2 regularization shrinks weights but rarely makes them zero.

C Both L1 and L2 regularization always set all weights to zero to prevent overfitting.

D L1 regularization increases model complexity, while L2 regularization decreases it.

💡 Hint

Think about which regularization method helps with feature selection by removing some features completely.

L1 regularization adds absolute value penalties and can zero out weights, creating sparse models. L2 adds squared penalties and shrinks weights but keeps them nonzero.

❓ Predict Output

intermediate

Output shape after applying L2 regularization in TensorFlow

What will be the shape of the weights after creating a Dense layer with 5 units and L2 regularization in TensorFlow?

TensorFlow

import tensorflow as tf
layer = tf.keras.layers.Dense(5, kernel_regularizer=tf.keras.regularizers.l2(0.01), input_shape=(10,))
model = tf.keras.Sequential([layer])
model.build()
weights_shape = model.layers[0].kernel.shape
print(weights_shape)

A (10, 5)

B (5, 10)

C (5,)

D (10,)

💡 Hint

Remember the Dense layer shape is (input_features, output_units).

The Dense layer's kernel weights shape is (input_dim, units). Here input_dim=10 and units=5, so shape is (10, 5).

❓ Hyperparameter

advanced

Choosing regularization strength

You train a neural network and notice it overfits the training data. Which change to the regularization parameter lambda (strength) will most likely reduce overfitting?

A Increase lambda to add stronger regularization.

B Decrease lambda to reduce regularization strength.

C Set lambda to zero to remove regularization.

D Keep lambda unchanged because it does not affect overfitting.

💡 Hint

Think about how regularization controls model complexity and overfitting.

Increasing lambda increases penalty on weights, encouraging simpler models and reducing overfitting.

❓ Metrics

advanced

Effect of L1 regularization on model sparsity metric

After training a model with L1 regularization, which metric would best show that many weights are exactly zero?

A Sum of all weights regardless of value.

B Mean squared error on training data.

C Count of weights equal to zero (sparsity count).

D Accuracy on validation data.

💡 Hint

Sparsity means many weights are zero. Which metric directly measures that?

Counting how many weights are exactly zero directly measures sparsity, which L1 regularization encourages.

🔧 Debug

expert

Identifying error in TensorFlow model with combined L1 and L2 regularization

Consider this TensorFlow code snippet that tries to apply both L1 and L2 regularization to a Dense layer. What error will it raise?

TensorFlow

import tensorflow as tf
layer = tf.keras.layers.Dense(4, kernel_regularizer=tf.keras.regularizers.l1_l2(l1=0.01, l2=0.01))
model = tf.keras.Sequential([layer])
model.build(input_shape=(None, 8))
print(model.layers[0].kernel_regularizer.l1)

A SyntaxError due to incorrect function call.

B TypeError because l1_l2 does not accept keyword arguments.

C No error; prints 0.01 as expected.

D AttributeError because 'kernel_regularizer' has no attribute 'l1'.

💡 Hint

Check the attributes available on the l1_l2 regularizer object.

The l1_l2 regularizer object does not expose 'l1' or 'l2' attributes directly, so accessing them causes AttributeError.