ResNet and skip connections in Computer Vision - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main problem ResNet aims to solve in deep neural networks?

ResNet addresses the problem of vanishing gradients and degradation in very deep networks, where adding more layers makes training harder and accuracy worse.

beginner

What is a skip connection in ResNet?

A skip connection is a shortcut that bypasses one or more layers by directly adding the input of those layers to their output, helping gradients flow better during training.

intermediate

How does a skip connection help during backpropagation?

Skip connections allow gradients to flow directly through the shortcut, reducing the chance of gradients becoming too small and enabling easier training of deep networks.

intermediate

In ResNet, what is the typical operation performed when combining the input and output in a skip connection?

The input is added element-wise to the output of the stacked layers, forming a residual block output: output = F(input) + input.

intermediate

Why are ResNet models often deeper than traditional CNNs?

Because skip connections help avoid training problems like vanishing gradients, ResNet can safely add many more layers, improving feature learning and accuracy.

What problem do skip connections in ResNet primarily address?

A Batch normalization

B Vanishing gradients

C Data augmentation

D Overfitting

Skip connections help gradients flow better, solving the vanishing gradient problem in deep networks.

In a ResNet residual block, how is the output computed?

A Output = F(input) + input

B Output = input * F(input)

C Output = F(input) - input

D Output = input / F(input)

The residual block output is the sum of the input and the function F(input) from the layers.

Why can ResNet have hundreds of layers without training issues?

A Because it uses dropout

B Because it uses small batch sizes

C Because of skip connections

D Because it uses ReLU activation only

Skip connections allow gradients to flow easily, enabling very deep networks to train well.

What does the 'F' represent in the ResNet formula output = F(input) + input?

A The identity function

B The input data

C The loss function

D The residual mapping learned by the layers

F is the function learned by the stacked layers in the residual block.

Which of these is NOT a benefit of skip connections?

A Increasing model overfitting

B Reduced training time

C Avoiding vanishing gradients

D Improved gradient flow

Skip connections help training but do not cause overfitting; they improve model learning.

Explain in your own words what a skip connection is and why it helps deep neural networks.

Think about how information flows in a network and what happens when it can skip some layers.

Describe the structure of a ResNet residual block and how it differs from a traditional convolutional block.

Focus on the addition operation and the shortcut path.