Dynamic computation graph advantage in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why use dynamic computation graphs in PyTorch?

Which of the following best explains the main advantage of dynamic computation graphs in PyTorch compared to static graphs?

A They allow changing the model structure during runtime, enabling flexible architectures like variable-length inputs.

B They automatically optimize the graph for faster execution without user intervention.

C They require less memory because the graph is precompiled once and reused for all inputs.

D They prevent any runtime errors by checking all operations before training starts.

💡 Hint

Think about how dynamic graphs let you build the model step-by-step as data flows through.

Dynamic computation graphs are built on-the-fly during each forward pass, allowing the model to change structure based on input data. This flexibility is useful for tasks like processing sequences of different lengths.

❓ Predict Output

intermediate

Output of dynamic graph example in PyTorch

What will be the output of the following PyTorch code snippet?

PyTorch

import torch
x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
y = x * 2 if x.sum() > 5 else x + 2
print(y)

A tensor([1., 2., 3.], requires_grad=True)

B tensor([3., 4., 5.], grad_fn=<AddBackward0>)

C RuntimeError due to conditional operation on tensor

D tensor([2., 4., 6.], grad_fn=<MulBackward0>)

💡 Hint

Check the sum of x and which branch of the if-else runs.

The sum of x is 6, which is greater than 5, so y = x * 2. The output tensor has gradient tracking because x requires grad.

❓ Model Choice

advanced

Choosing model type for variable-length input sequences

You want to build a model that processes sentences of different lengths without padding. Which model type benefits most from dynamic computation graphs?

A Feedforward Neural Network with fixed input dimension

B Convolutional Neural Network (CNN) with fixed input size

C Recurrent Neural Network (RNN) implemented in PyTorch

D Support Vector Machine (SVM) with fixed feature vectors

💡 Hint

Think about models that naturally handle sequences of varying lengths.

RNNs process inputs step-by-step and can handle variable-length sequences naturally. Dynamic graphs in PyTorch allow building the graph differently for each sequence length.

❓ Hyperparameter

advanced

Effect of dynamic graph on training speed

How does using a dynamic computation graph in PyTorch typically affect training speed compared to static graphs?

A Training is always faster because dynamic graphs optimize execution automatically.

B Training is usually slower because the graph is rebuilt every iteration.

C Training speed is unaffected because graphs are cached after first build.

D Training speed depends only on hardware, not graph type.

💡 Hint

Consider the overhead of building the graph each time.

Dynamic graphs are rebuilt every forward pass, which adds overhead and can slow training compared to static graphs that are compiled once.

🔧 Debug

expert

Identifying error in dynamic graph usage

What error will this PyTorch code raise when run?

import torch
x = torch.tensor([1.0, 2.0], requires_grad=True)
with torch.no_grad():
    y = x * 2
z = y.sum()
z.backward()

A RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

B No error, backward runs successfully

C TypeError: unsupported operand type(s) for *: 'Tensor' and 'int'

D ValueError: expected scalar for backward()

💡 Hint

Think about what torch.no_grad() does to the computation graph.

Inside torch.no_grad(), operations do not track gradients. So y does not require grad, causing backward() to fail with RuntimeError.