Linear (fully connected) layers in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a linear (fully connected) layer in a neural network?

A linear layer connects every input neuron to every output neuron with a weight. It performs a weighted sum of inputs plus a bias, producing outputs.

beginner

How do you create a linear layer in PyTorch?

Use torch.nn.Linear(input_size, output_size). For example, torch.nn.Linear(10, 5) creates a layer with 10 inputs and 5 outputs.

beginner

What are the learnable parameters in a linear layer?

The weights matrix and the bias vector. Weights connect inputs to outputs, and bias shifts the output values.

intermediate

What is the shape of the weight matrix in a linear layer with input size 8 and output size 3?

The weight matrix shape is (3, 8). It has one row per output and one column per input.

beginner

Why do we add a bias term in a linear layer?

Bias allows the model to shift the output up or down, helping it fit data better by not forcing the output to pass through zero.

What does a linear layer do in a neural network?

A Reduces the size of the input data by half

B Computes a weighted sum of inputs plus bias

C Applies a non-linear activation function

D Normalizes the input data

A linear layer multiplies inputs by weights and adds bias to produce outputs.

In PyTorch, which class creates a linear layer?

A torch.nn.ReLU

B torch.nn.Conv2d

C torch.nn.Linear

D torch.nn.BatchNorm1d

torch.nn.Linear creates a fully connected linear layer.

If a linear layer has input size 4 and output size 2, what is the shape of its weight matrix?

A (4, 2)

B (2, 2)

C (4, 4)

D (2, 4)

Weight matrix shape is (output_size, input_size), so (2, 4).

What is the role of the bias in a linear layer?

A To shift the output values

B To reduce overfitting

C To normalize the output

D To scale the inputs

Bias shifts the output values, allowing more flexible fitting.

Which of these is NOT true about linear layers?

A They apply a non-linear activation by default

B They include weights and bias parameters

C They connect every input to every output

D They perform a matrix multiplication

Linear layers do NOT apply non-linear activation by default.

Explain how a linear (fully connected) layer transforms input data in a neural network.

Think about multiplying inputs by weights and adding bias.

Describe how you would implement a linear layer in PyTorch and what parameters it needs.

Focus on the class name and constructor arguments.