GradientTape for automatic differentiation in TensorFlow - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to record operations for automatic differentiation using GradientTape.

TensorFlow

import tensorflow as tf
x = tf.Variable(3.0)
with tf.GradientTape() as tape:
    y = x * x
grad = tape.[1](y, x)

Drag options to blanks, or click blank then click option'

Agradient

Bgradients

Cgrad

Ddifferentiate

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'gradients' instead of 'gradient' causes an AttributeError.

Trying to call 'grad' or 'differentiate' which do not exist.

The method to compute gradients in TensorFlow's GradientTape is gradient.

2fill in blank

medium

Complete the code to watch a tensor for gradient calculation inside GradientTape.

TensorFlow

import tensorflow as tf
x = tf.constant(4.0)
tape = tf.GradientTape()
tape.[1](x)
y = x * x
grad = tape.gradient(y, x)

Drag options to blanks, or click blank then click option'

Amonitor

Bobserve

Ctrack

Dwatch

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'observe' or 'track' which are not valid methods.

Not calling watch on constant tensors causes no gradient to be computed.

To tell GradientTape to watch a tensor, use the watch method.

3fill in blank

hard

Fix the error in the code to correctly compute the gradient of y = x^3.

TensorFlow

import tensorflow as tf
x = tf.Variable(2.0)
with tf.GradientTape() as tape:
    y = x[1]3
grad = tape.gradient(y, x)

Drag options to blanks, or click blank then click option'

A^

B*

C**

D//

Attempts:

3 left

💡 Hint

Common Mistakes

Using '^' causes wrong output because it is bitwise XOR.

Using '*' repeats multiplication but does not raise to power.

In Python, exponentiation is done with **, not ^.

4fill in blank

hard

Fill both blanks to create a GradientTape context and compute the gradient of z = x * y.

TensorFlow

import tensorflow as tf
x = tf.Variable(3.0)
y = tf.Variable(4.0)
with tf.GradientTape([1]=True) as tape:
    z = x [2] y

Drag options to blanks, or click blank then click option'

Awatch_accessed_variables

Bwatch

C*

D+

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'watch' as a parameter name causes an error.

Using '+' instead of '*' changes the operation.

Setting watch_accessed_variables=True makes tape watch variables automatically. The multiplication operator is *.

5fill in blank

hard

Fill all three blanks to compute the gradient of f = x^2 + y^2 with respect to x and y.

TensorFlow

import tensorflow as tf
x = tf.Variable(1.0)
y = tf.Variable(2.0)
with tf.GradientTape() as tape:
    f = x[1]2 + y[2]2
gradients = tape.gradient(f, [3])

Drag options to blanks, or click blank then click option'

A**

B*

C[x, y]

D(x, y)

Attempts:

3 left

💡 Hint

Common Mistakes

Using '*' instead of '**' for power.

Passing variables as a tuple instead of a list.

Use ** for power, and pass a list of variables [x, y] to compute gradients for both.

Practice

1. What is the main purpose of TensorFlow's GradientTape?

easy

A. To automatically compute derivatives of operations for optimization

B. To visualize the model architecture

C. To load datasets for training

D. To save the trained model to disk

Solution

1. Step 1: Understand what GradientTape records

   GradientTape records operations on tensors to compute gradients automatically.

2. Step 2: Connect gradients to optimization

   These gradients help update model parameters during training to minimize loss.

3. Final Answer:

   To automatically compute derivatives of operations for optimization -> Option A

4. Quick Check:

   GradientTape = automatic differentiation [OK]

Hint: GradientTape records ops to find derivatives fast [OK]

Common Mistakes:

• Thinking GradientTape is for data loading
• Confusing it with model saving functions
• Assuming it visualizes model structure

2. Which of the following is the correct way to start recording operations with GradientTape in TensorFlow?

easy

A. GradientTape.record()

B. tf.GradientTape.start()

C. with tf.GradientTape() as tape:

D. tf.GradientTape.begin()

Solution

1. Step 1: Recall the syntax for using GradientTape

   GradientTape is used as a context manager with the with statement.

2. Step 2: Identify the correct syntax

   The correct syntax is with tf.GradientTape() as tape: to start recording.

3. Final Answer:

   with tf.GradientTape() as tape: -> Option C

4. Quick Check:

   Use 'with' block for GradientTape [OK]

Hint: GradientTape always used inside a 'with' block [OK]

Common Mistakes:

• Trying to call start() or begin() methods which don't exist
• Not using 'with' statement
• Using incorrect method names like record()

3. What will be the output of this code snippet?

import tensorflow as tf
x = tf.constant(3.0)
with tf.GradientTape() as tape:
    tape.watch(x)
    y = x * x + 2 * x + 1
dy_dx = tape.gradient(y, x)
print(dy_dx.numpy())

medium

A. 9.0

B. 11.0

C. 5.0

D. 8.0

Solution

1. Step 1: Analyze the function

   The code defines y = x*x + 2*x + 1 with x = tf.constant(3.0).

2. Step 2: Compute the derivative

   The derivative dy/dx = 2*x + 2. At x=3.0, this is 2*3 + 2 = 8.0.

3. Step 3: Verify code execution

   tape.watch(x) enables gradient tracking. tape.gradient(y, x) returns 8.0, printed via .numpy().

4. Final Answer:

   8.0 -> Option D

5. Quick Check:

   2*3 + 2 = 8.0 [OK]

Hint: Derivative of x²+2x+1 is 2x+2; plug in x=3 [OK]

Common Mistakes:

• Forgetting to watch the tensor x
• Miscomputing the derivative formula
• Confusing y with x in gradient call

4. Identify the error in this TensorFlow code using GradientTape:

import tensorflow as tf
x = tf.constant(2.0)
with tf.GradientTape() as tape:
    y = x * x
dy_dx = tape.gradient(y, x)
print(dy_dx.numpy())

medium

A. x is not watched, so gradient is None

B. GradientTape must be used without 'with' block

C. y should be defined outside the tape block

D. tape.gradient requires a list of variables

Solution

1. Step 1: Check if x is watched

   x is a constant and not watched by default, so gradient will be None.

2. Step 2: Understand how to fix

   Use tape.watch(x) to track x for gradient calculation.

3. Final Answer:

   x is not watched, so gradient is None -> Option A

4. Quick Check:

   Must watch constants for gradients [OK]

Hint: Watch constants explicitly to get gradients [OK]

Common Mistakes:

• Assuming constants are watched automatically
• Misusing GradientTape without 'with' block
• Passing wrong arguments to gradient()

5. You want to compute gradients of a function f(x, y) = x^2 * y + y^3 with respect to both x and y. Which code snippet correctly computes both gradients using TensorFlow's GradientTape?

hard

A. with tf.GradientTape() as tape:
x = tf.constant(2.0)
y = tf.constant(3.0)
f = x**2 * y + y**3
grads = tape.gradient(f, x, y)
print(grads)

B. with tf.GradientTape() as tape:
x = tf.constant(2.0)
y = tf.constant(3.0)
tape.watch([x, y])
f = x**2 * y + y**3
grads = tape.gradient(f, [x, y])
print(grads)

C. with tf.GradientTape() as tape:
x = tf.constant(2.0)
y = tf.constant(3.0)
tape.watch(x)
f = x**2 * y + y**3
grads = tape.gradient(f, y)
print(grads)

D. with tf.GradientTape() as tape:
x = tf.constant(2.0)
y = tf.constant(3.0)
tape.watch(x)
tape.watch(y)
f = x**2 * y + y**3
grads = tape.gradient(f, [x, y])
print(grads)

Solution

1. Step 1: Understand watching variables

   Constants must be watched explicitly to compute gradients.

2. Step 2: Check gradient call syntax

   GradientTape.gradient expects a list of variables to compute gradients for multiple variables.

3. Step 3: Compare options

   with tf.GradientTape() as tape: x = tf.constant(2.0) y = tf.constant(3.0) tape.watch([x, y]) f = x**2 * y + y**3 grads = tape.gradient(f, [x, y]) print(grads) calls tape.watch([x, y]) once with a list, then passes [x, y] to gradient().

4. Final Answer:

   Explicitly watch both x and y together and pass [x, y] to tape.gradient(f, [x, y]) -> Option B

5. Quick Check:

   Watch all variables as list and pass as list to gradient() [OK]

Hint: Watch all variables and pass as list to gradient() [OK]

Common Mistakes:

• Not watching all variables
• Passing variables separately instead of as list
• Computing gradient for only one variable