Dropout (nn.Dropout) in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of Dropout layer during training

What is the output of the following PyTorch code snippet when the model is in training mode?

PyTorch

import torch
import torch.nn as nn

torch.manual_seed(0)
dropout = nn.Dropout(p=0.5)
input_tensor = torch.tensor([1.0, 2.0, 3.0, 4.0])
output = dropout(input_tensor)
print(output)

A tensor([2., 0., 6., 8.])

B tensor([1., 2., 3., 4.])

C tensor([0., 0., 0., 0.])

D tensor([0.5, 1., 1.5, 2.])

💡 Hint

Remember that dropout randomly zeroes some elements and scales the others by 1/(1-p) during training.

During training, nn.Dropout with p=0.5 randomly sets half of the input elements to zero and scales the remaining elements by 1/(1-0.5) = 2. Given the fixed seed, the output tensor is [2., 0., 6., 8.].

🧠 Conceptual

intermediate

Effect of Dropout during evaluation

What happens to the output of nn.Dropout when the model is switched to evaluation mode (model.eval())?

A Dropout randomly zeros some inputs and scales the rest.

B Dropout sets all inputs to zero.

C Dropout scales the input by the dropout probability p.

D Dropout passes the input through unchanged without any modification.

💡 Hint

Think about why dropout is only used during training.

During evaluation, dropout is disabled and acts as an identity function, passing inputs unchanged to ensure consistent predictions.

❓ Hyperparameter

advanced

Choosing dropout probability

Which dropout probability (p) value is generally recommended to prevent overfitting without causing underfitting in a neural network?

A p = 0.5 (moderate dropout)

B p = 0.1 (very low dropout)

C p = 0.0 (no dropout)

D p = 0.9 (very high dropout)

💡 Hint

Typical dropout values are between 0.2 and 0.5.

A dropout probability around 0.5 is commonly used to balance regularization and model capacity, reducing overfitting without hurting learning too much.

🔧 Debug

advanced

Identifying error in dropout usage

What error will occur when running this code snippet?

PyTorch

import torch
import torch.nn as nn

dropout = nn.Dropout(p=1.2)
input_tensor = torch.tensor([1.0, 2.0, 3.0])
output = dropout(input_tensor)
print(output)

A ValueError: input tensor shape mismatch

B RuntimeError: dropout probability has to be between 0 and 1

C No error, outputs tensor with all zeros

D TypeError: input_tensor must be float type

💡 Hint

Check the valid range for dropout probability p.

Dropout probability p must be between 0 and 1 inclusive. Setting p=1.2 causes a runtime error.

❓ Model Choice

expert

Best place to apply dropout in a neural network

Where is the best place to apply nn.Dropout in a feedforward neural network to improve generalization?

A After the output layer

B Before the input layer

C Between hidden layers after activation functions

D Only on the final prediction output

💡 Hint

Dropout is usually applied to internal layers to prevent co-adaptation of neurons.

Applying dropout between hidden layers after activations helps regularize the network by randomly dropping neurons during training.