Dropout (nn.Dropout) in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of Dropout in neural networks?

Dropout helps prevent overfitting by randomly turning off some neurons during training. This forces the network to learn more robust features.

beginner

How does nn.Dropout behave differently during training and evaluation in PyTorch?

During training, nn.Dropout randomly sets some inputs to zero. During evaluation, it passes all inputs unchanged to keep the full signal.

beginner

What does the parameter 'p' in nn.Dropout(p) represent?

The parameter 'p' is the probability of an element being zeroed (dropped) during training. For example, p=0.3 means 30% of neurons are dropped randomly.

intermediate

Show a simple PyTorch example of applying nn.Dropout with p=0.5 to a tensor.

import torch
import torch.nn as nn

x = torch.ones(5)
dropout = nn.Dropout(p=0.5)
dropout.train()  # set to training mode
output = dropout(x)
print(output)

# Output will have about half the elements set to zero randomly.

beginner

Why is Dropout usually turned off during model evaluation?

Dropout is turned off during evaluation to use the full network capacity for predictions. Turning it off ensures stable and consistent outputs.

What does nn.Dropout(p=0.2) do during training?

A Randomly sets 20% of inputs to zero

B Randomly sets 80% of inputs to zero

C Scales inputs by 0.2

D Does nothing

The parameter p=0.2 means 20% of the inputs are randomly dropped (set to zero) during training.

During evaluation, nn.Dropout in PyTorch:

A Randomly drops neurons

B Scales outputs by dropout probability

C Raises an error

D Passes inputs unchanged

Dropout is disabled during evaluation, so inputs pass through unchanged.

Why use Dropout in training a neural network?

A To speed up training

B To increase model size

C To prevent overfitting

D To reduce input size

Dropout helps prevent overfitting by making the network less reliant on any single neuron.

If you set nn.Dropout(p=0), what happens?

A All neurons are dropped

B No neurons are dropped

C Half neurons are dropped

D Error occurs

p=0 means zero probability of dropping neurons, so dropout does nothing.

Which mode must be set for nn.Dropout to actually drop neurons?

A train()

B eval()

C test()

D dropout()

Dropout only drops neurons when the module is in training mode.

Explain how nn.Dropout works and why it is useful in training neural networks.

Think about how turning off some neurons helps the network learn better.

Describe the difference in behavior of nn.Dropout during training and evaluation phases.

Consider what happens to the inputs in each phase.