Dense (fully connected) layers in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output shape of a Dense layer

Given the following TensorFlow code, what is the shape of the output tensor after the Dense layer?

TensorFlow

import tensorflow as tf

input_tensor = tf.random.uniform((5, 10))  # batch size 5, input features 10
layer = tf.keras.layers.Dense(8)
output = layer(input_tensor)
output_shape = output.shape
print(output_shape)

A (10, 8)

B (5, 8)

C (5, 10)

D (8, 5)

💡 Hint

Remember the Dense layer changes the last dimension to the number of units.

The Dense layer transforms each input vector of size 10 into a vector of size 8. Since the batch size is 5, the output shape is (5, 8).

❓ Model Choice

intermediate

Choosing the correct Dense layer for classification

You want to build a neural network to classify images into 10 categories. Which Dense layer configuration is most appropriate for the final layer?

A Dense(10, activation='softmax')

B Dense(1, activation='sigmoid')

C Dense(10, activation='relu')

D Dense(5, activation='softmax')

💡 Hint

For multi-class classification, the output layer should have one unit per class with softmax activation.

For 10 classes, the output layer should have 10 units with softmax activation to output probabilities for each class.

❓ Hyperparameter

advanced

Effect of changing units in Dense layer

What is the effect of increasing the number of units in a Dense layer from 32 to 128 in a neural network?

A Increases model capacity and may improve learning but risks overfitting

B Decreases model capacity and speeds up training

C Has no effect on model capacity or training time

D Always guarantees better accuracy on test data

💡 Hint

More units mean more parameters to learn.

Increasing units increases the number of parameters, allowing the model to learn more complex patterns but also increasing overfitting risk.

🔧 Debug

advanced

Identifying error in Dense layer usage

What error will this code raise when run?

TensorFlow

import tensorflow as tf

input_tensor = tf.random.uniform((5, 10))
layer = tf.keras.layers.Dense(-5)
output = layer(input_tensor)

A RuntimeError: layer not built

B TypeError: input_tensor shape invalid

C No error, runs successfully

D ValueError: units must be a positive integer

💡 Hint

Check the units argument for Dense layer.

Dense layer units must be a positive integer. Negative units cause a ValueError.

🧠 Conceptual

expert

Why use bias in Dense layers?

What is the main purpose of including a bias term in a Dense (fully connected) layer?

A To reduce the number of parameters in the model

B To normalize the input data automatically

C To allow the model to fit data that does not pass through the origin

D To prevent overfitting by adding regularization

💡 Hint

Think about shifting the activation function.

Bias allows the activation function to shift, enabling the model to fit data that does not pass through zero.

Practice

1. What does a Dense (fully connected) layer do in a neural network?

easy

A. Does not connect any neurons, only passes data through

B. Connects every input neuron to every output neuron with weights

C. Connects neurons randomly without weights

D. Only connects input neurons to output neurons with zero weights

Solution

1. Step 1: Understand the role of Dense layers

   A Dense layer connects each input neuron to every output neuron using weights and biases to learn patterns.

2. Step 2: Compare options with Dense layer behavior

   Only Connects every input neuron to every output neuron with weights correctly describes this full connection with weights; others are incorrect or incomplete.

3. Final Answer:

   Connects every input neuron to every output neuron with weights -> Option B

4. Quick Check:

   Dense layer = full weighted connections [OK]

Hint: Dense means all inputs connect to all outputs [OK]

Common Mistakes:

• Thinking Dense layers connect neurons randomly
• Believing Dense layers have zero weights
• Assuming Dense layers do not connect neurons

2. Which of the following is the correct way to add a Dense layer with 10 neurons and ReLU activation in TensorFlow?

easy

A. tf.keras.layers.Dense(10, activation='relu')

B. tf.keras.DenseLayer(10, activation='relu')

C. tf.layers.Dense(activation='relu', units=10)

D. tf.keras.layers.Dense(activation='relu', neurons=10)

Solution

1. Step 1: Recall TensorFlow Dense layer syntax

   The correct syntax is tf.keras.layers.Dense(units, activation='function').

2. Step 2: Match options to correct syntax

   tf.keras.layers.Dense(10, activation='relu') matches this exactly. Others have wrong class names or parameter names.

3. Final Answer:

   tf.keras.layers.Dense(10, activation='relu') -> Option A

4. Quick Check:

   Correct Dense syntax = tf.keras.layers.Dense(10, activation='relu') [OK]

Hint: Use tf.keras.layers.Dense(units, activation) [OK]

Common Mistakes:

• Using wrong class name like DenseLayer
• Swapping parameter names (neurons vs units)
• Placing activation before units

3. What will be the output shape of this model?

model = tf.keras.Sequential([
  tf.keras.layers.Dense(5, input_shape=(3,)),
  tf.keras.layers.Dense(2)
])
output = model(tf.constant([[1.0, 2.0, 3.0]]))
print(output.shape)

medium

A. (3, 2)

B. (1, 5)

C. (1, 2)

D. (3, 5)

Solution

1. Step 1: Analyze model layers and input shape

   Input shape is (3,), first Dense outputs 5 units, second Dense outputs 2 units.

2. Step 2: Determine output shape after second Dense

   Batch size is 1 (one input), final output shape is (1, 2).

3. Final Answer:

   (1, 2) -> Option C

4. Quick Check:

   Output shape = (batch_size, last layer units) = (1, 2) [OK]

Hint: Output shape = (batch, last Dense units) [OK]

Common Mistakes:

• Confusing input shape with output shape
• Mixing up units of first and second Dense layers
• Ignoring batch dimension

4. Identify the error in this code snippet:

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(10, input_shape=(4,)))
model.add(tf.keras.layers.Dense(5, activation='relu'))
model.add(tf.keras.layers.Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit(x_train, y_train, epochs=5)

medium

A. Loss function 'mse' is invalid

B. Input shape should be specified in the first layer only

C. Missing activation in the first Dense layer

D. No error, code is correct

Solution

1. Step 1: Check Dense layer usage and input shape

   Input shape is correctly specified in the first Dense layer only.

2. Step 2: Verify loss function and activation usage

   Loss 'mse' is valid for regression; activation in second layer is fine; first layer activation is optional.

3. Final Answer:

   No error, code is correct -> Option D

4. Quick Check:

   Code syntax and usage are correct [OK]

Hint: Input shape only in first layer; 'mse' is valid loss [OK]

Common Mistakes:

• Thinking activation is mandatory in every Dense layer
• Specifying input_shape in multiple layers
• Believing 'mse' is invalid loss

5. You want to build a model to classify images into 3 categories. Which Dense layer setup is best for the output layer?

hard

A. Dense(3, activation='softmax')

B. Dense(1, activation='sigmoid')

C. Dense(3, activation='relu')

D. Dense(3)

Solution

1. Step 1: Understand classification output needs

   For 3 categories, output layer should have 3 units, one per class.

2. Step 2: Choose activation for multi-class classification

   Softmax activation outputs probabilities summing to 1, ideal for multi-class.

3. Step 3: Evaluate options

   Dense(3, activation='softmax') uses 3 units with softmax, perfect for 3-class classification; others are unsuitable.

4. Final Answer:

   Dense(3, activation='softmax') -> Option A

5. Quick Check:

   Multi-class output = units=classes + softmax [OK]

Hint: Use softmax with units = number of classes [OK]

Common Mistakes:

• Using sigmoid for multi-class output
• Omitting activation in output layer
• Using relu activation for output