Why regularization prevents overfitting in TensorFlow - Quick Recap

Recall & Review

beginner

What is overfitting in machine learning?

Overfitting happens when a model learns the training data too well, including noise and details that don't apply to new data. This makes the model perform poorly on unseen data.

beginner

What does regularization do in a machine learning model?

Regularization adds a penalty to the model's complexity, encouraging it to keep weights small and simple. This helps the model focus on the main patterns and avoid fitting noise.

intermediate

How does L2 regularization (weight decay) work?

L2 regularization adds the sum of squared weights to the loss function. This pushes the model to keep weights smaller, which reduces overfitting by making the model simpler.

intermediate

Why does keeping model weights small help prevent overfitting?

Small weights mean the model changes less for small input changes, making it less sensitive to noise and more focused on general patterns.

beginner

Name two common types of regularization used in TensorFlow.

Two common types are L1 regularization (which encourages sparsity) and L2 regularization (which encourages small weights). Both help reduce overfitting.

What problem does regularization mainly help to solve?

A Model deployment

B Underfitting

C Data collection

D Overfitting

Regularization helps prevent overfitting by keeping the model simpler and less sensitive to noise.

Which of these is a common regularization technique?

A L2 regularization

B Batch normalization

C Dropout

D Data augmentation

L2 regularization adds a penalty on large weights to reduce overfitting.

What effect does L2 regularization have on model weights?

A Makes weights larger

B Removes weights

C Makes weights smaller

D Duplicates weights

L2 regularization encourages smaller weights to simplify the model.

Why is a simpler model less likely to overfit?

A It ignores all data

B It focuses on main patterns, not noise

C It memorizes training data

D It uses more layers

Simpler models focus on general patterns and avoid fitting noise in training data.

Which regularization method encourages sparsity in weights?

A L1 regularization

B L2 regularization

C Dropout

D Early stopping

L1 regularization encourages many weights to become zero, creating sparsity.

Explain in your own words why regularization helps prevent overfitting in machine learning models.

Think about how adding a penalty to big weights changes the model's behavior.

Describe the difference between L1 and L2 regularization and how each affects the model.

Consider how each penalty changes the weights during training.