GPU vs CPU tensor placement in TensorFlow - Practice Questions

Challenge - 5 Problems

❓ Predict Output

intermediate

Tensor device placement output

What device will the tensor be placed on and what will be the output of the following code snippet?

TensorFlow

import tensorflow as tf

with tf.device('/CPU:0'):
    a = tf.constant([1.0, 2.0, 3.0])

with tf.device('/GPU:0'):
    b = tf.constant([4.0, 5.0, 6.0])

print(a.device)
print(b.device)

A a.device shows GPU device string, b.device shows CPU device string

B Both a.device and b.device show CPU device string

C a.device shows CPU device string, b.device shows GPU device string

D Both a.device and b.device show GPU device string

💡 Hint

Tensors are placed on the device specified by the tf.device context manager.

The tensor 'a' is created inside the CPU device context, so it is placed on CPU. The tensor 'b' is created inside the GPU device context, so it is placed on GPU. The device strings printed reflect this placement.

🧠 Conceptual

intermediate

Tensor operations device fallback

If you create a tensor on GPU but perform an operation with a tensor on CPU, where will TensorFlow perform the operation?

A On CPU, because one tensor is on CPU

B On GPU, because one tensor is on GPU

C TensorFlow raises an error due to device mismatch

D TensorFlow automatically copies tensors to the same device and performs operation there

💡 Hint

TensorFlow tries to perform operations on the same device by copying tensors if needed.

TensorFlow automatically copies tensors to the same device before performing operations. So if tensors are on different devices, it copies one to the other's device and then performs the operation.

❓ Metrics

advanced

Comparing training speed on CPU vs GPU

You train the same neural network model on CPU and GPU. Which metric difference best indicates GPU acceleration?

A Lower training time per epoch on GPU than CPU

B Higher memory usage on CPU than GPU

C Lower validation accuracy on GPU than CPU

D Higher training loss on GPU than CPU

💡 Hint

GPU is designed to speed up computations, so training time is key.

GPU acceleration typically reduces training time per epoch compared to CPU. Loss or accuracy differences are not direct indicators of device speed.

🔧 Debug

advanced

TensorFlow device placement error

What error will this code raise and why? import tensorflow as tf with tf.device('/GPU:1'): x = tf.constant([1, 2, 3])

A No error, tensor placed on GPU:1

B RuntimeError: GPU device '/GPU:1' not found

C SyntaxError: invalid syntax in device string

D ValueError: Tensor shape mismatch

💡 Hint

Check if the specified GPU device exists on your machine.

If the machine has only one GPU (usually '/GPU:0'), specifying '/GPU:1' causes a runtime error because that device does not exist.

❓ Model Choice

expert

Choosing device placement for large model training

You have a large deep learning model that does not fit into GPU memory. Which strategy is best to train it efficiently?

A Use model parallelism to split model across multiple GPUs

B Place entire model on CPU to avoid GPU memory limits

C Reduce batch size and place model on single GPU

D Convert model to float16 and place on CPU

💡 Hint

Splitting model across GPUs helps handle large models exceeding single GPU memory.

Model parallelism splits the model across multiple GPUs, allowing training of large models that don't fit on one GPU. Placing on CPU is slower, reducing batch size may not be enough, and float16 on CPU doesn't solve memory or speed issues.