Dynamic computation graph advantage in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a tensor with gradient tracking enabled.

PyTorch

import torch
x = torch.tensor([1.0, 2.0, 3.0], requires_grad=[1])

Drag options to blanks, or click blank then click option'

ANone

BTrue

CFalse

D0

Attempts:

3 left

💡 Hint

Common Mistakes

Setting requires_grad to False will not track gradients.

Using None or 0 will cause errors or no gradient tracking.

Setting requires_grad=True tells PyTorch to track operations on the tensor for automatic differentiation.

2fill in blank

medium

Complete the code to perform a dynamic computation where the number of operations depends on input size.

PyTorch

import torch
x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
result = 0
for i in range(x.size(0)):
    result += x[i] * [1]

Drag options to blanks, or click blank then click option'

A2

Bx

Ci

Dresult

Attempts:

3 left

💡 Hint

Common Mistakes

Using a fixed number like 2 makes the graph static.

Using the whole tensor x or result causes errors or wrong calculations.

Using i inside the loop makes the computation graph dynamic, changing with input size.

3fill in blank

hard

Fix the error in the code to correctly compute gradients with a dynamic graph.

PyTorch

import torch
x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
result = 0
for i in range(x.size(0)):
    result += x[i] * i
result.[1]()

Drag options to blanks, or click blank then click option'

Abackward

Bbackprop

Cgrad

Dcompute_grad

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'grad' or 'backprop' are not valid PyTorch methods.

Not calling backward() means gradients won't be computed.

The backward() function computes gradients for tensors with requires_grad=True.

4fill in blank

hard

Fill both blanks to create a dynamic computation graph that sums squares of input elements greater than 2.

PyTorch

import torch
x = torch.tensor([1.0, 3.0, 4.0], requires_grad=True)
result = 0
for val in x:
    if val [1] 2:
        result += val [2] 2
result.backward()

Drag options to blanks, or click blank then click option'

A>

B**

C*

D<

Attempts:

3 left

💡 Hint

Common Mistakes

Using '<' instead of '>' changes the condition.

Using '*' instead of '**' multiplies by 2 instead of squaring.

The condition val > 2 selects elements greater than 2, and val ** 2 squares them.

5fill in blank

hard

Fill all three blanks to build a dynamic graph that creates a dictionary of squares for even numbers in a list.

PyTorch

numbers = [1, 2, 3, 4, 5]
squares = { [1]: [2] for [3] in numbers if [3] % 2 == 0 }

Drag options to blanks, or click blank then click option'

Anum

Bnum ** 2

Dn

Attempts:

3 left

💡 Hint

Common Mistakes

Using different variable names inconsistently causes errors.

Using multiplication instead of exponentiation changes the result.

We use num as the key and num ** 2 as the value for even numbers iterated by num.

Practice

1. What is the main advantage of a dynamic computation graph in PyTorch?

easy

A. It allows the model to change its structure during runtime based on input data.

B. It requires pre-defining the entire model graph before training.

C. It makes the model run faster by using static graphs.

D. It prevents any changes to the model once training starts.

Solution

1. Step 1: Understand dynamic graph behavior

   Dynamic graphs build the model step-by-step as data flows, allowing changes during runtime.

2. Step 2: Compare with static graphs

   Static graphs require the entire model to be fixed before running, unlike dynamic graphs.

3. Final Answer:

   It allows the model to change its structure during runtime based on input data. -> Option A

4. Quick Check:

   Dynamic graph = flexible model structure [OK]

Hint: Dynamic graphs let models adapt during runtime [OK]

Common Mistakes:

• Confusing dynamic graphs with static graphs
• Thinking model structure is fixed before training
• Assuming dynamic graphs always run faster

2. Which PyTorch code snippet correctly creates a dynamic computation graph with gradient tracking for a simple addition?

easy

A. x = torch.tensor(2); y = torch.tensor(3); z = x + y

B. x = torch.tensor(2, graph_static=True); y = torch.tensor(3, graph_static=True); z = x + y

C. x = torch.tensor(2); y = torch.tensor(3); z = x * y

D. x = torch.tensor(2, requires_grad=True); y = torch.tensor(3, requires_grad=True); z = x + y

Solution

1. Step 1: Identify dynamic graph creation

   In PyTorch, setting requires_grad=True enables tracking operations for dynamic graph.

2. Step 2: Check code correctness

   x = torch.tensor(2, requires_grad=True); y = torch.tensor(3, requires_grad=True); z = x + y correctly sets requires_grad=True for tensors and adds them, building dynamic graph.

3. Final Answer:

   x = torch.tensor(2, requires_grad=True); y = torch.tensor(3, requires_grad=True); z = x + y -> Option D

4. Quick Check:

   requires_grad=True enables dynamic graph [OK]

Hint: Use requires_grad=True to track operations [OK]

Common Mistakes:

• Missing requires_grad=True disables graph tracking
• Using graph_static=True which is invalid
• Confusing multiplication with addition

3. What will be the output of this PyTorch code snippet?

import torch
x = torch.tensor(1.0, requires_grad=True)
y = x * 2
if y.item() > 1.5:
    z = y * 3
else:
    z = y * 4
z.backward()
print(x.grad.item())

medium

A. 3.0

B. 6.0

C. 4.0

D. 2.0

Solution

1. Step 1: Trace the computation graph

   x=1.0 with grad tracking; y = x*2 = 2.0; since y>1.5, z = y*3 = 6.0

2. Step 2: Compute gradient with backward()

   dz/dx = dz/dy * dy/dx = 3 * 2 = 6; but PyTorch accumulates gradients, so x.grad = 6

3. Final Answer:

   6.0 -> Option B

4. Quick Check:

   Gradient = 6.0 [OK]

Hint: Follow operations and chain rule for gradients [OK]

Common Mistakes:

• Ignoring the if condition effect
• Confusing gradient with output value
• Forgetting to call backward()

4. Identify the error in this PyTorch code that tries to use dynamic computation graph:

import torch
x = torch.tensor(2.0)
y = x * 3
z = y ** 2
z.backward()
print(x.grad)

medium

A. x tensor is missing requires_grad=True, so gradient is None.

B. Backward() is called on a non-scalar tensor causing error.

C. The power operator ** is not supported in PyTorch tensors.

D. The print statement syntax is incorrect.

Solution

1. Step 1: Check requires_grad flag

   x is created without requires_grad=True, so no graph tracks operations on x.

2. Step 2: Effect on backward()

   Calling backward() on z raises RuntimeError because no computation graph is tracked without requires_grad=True on x.

3. Final Answer:

   x tensor is missing requires_grad=True, so gradient is None. -> Option A

4. Quick Check:

   requires_grad=True needed for gradients [OK]

Hint: Always set requires_grad=True to track gradients [OK]

Common Mistakes:

• Assuming backward() works without requires_grad
• Thinking ** operator is invalid
• Confusing print syntax errors

5. You want to build a PyTorch model that changes its forward pass based on input size dynamically. Which feature of dynamic computation graphs helps achieve this?

hard

A. Fixing the model architecture to a single shape for all inputs.

B. Pre-compiling the entire model graph before training starts.

C. Ability to define the computation graph on-the-fly during each forward pass.

D. Using static graph optimizations for faster inference.

Solution

1. Step 1: Understand dynamic graph flexibility

   Dynamic graphs let you build the computation graph during each forward pass, adapting to input size.

2. Step 2: Contrast with static graphs

   Static graphs require fixed architecture, so they can't adapt dynamically to input changes.

3. Final Answer:

   Ability to define the computation graph on-the-fly during each forward pass. -> Option C

4. Quick Check:

   Dynamic graph = on-the-fly graph building [OK]

Hint: Dynamic graphs build model each forward pass [OK]

Common Mistakes:

• Confusing dynamic with static graph behavior
• Thinking model must be fixed before training
• Assuming static graphs allow input size changes