Multi-input and multi-output models in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Multi-input and multi-output models

Problem:You have a model that takes two different inputs: one numeric vector and one image. It predicts two outputs: a continuous value and a category label. The current model trains but shows low accuracy on the classification output and high loss on the regression output.

Current Metrics:Regression loss: 0.8, Classification accuracy: 60%

Issue:The model is not learning well from both inputs together and the outputs are not accurate enough.

Your Task

Improve the model so that classification accuracy is above 75% and regression loss is below 0.4.

Keep the model architecture as multi-input and multi-output.

Do not change the dataset or input data preprocessing.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models, Input

# Numeric input branch
numeric_input = Input(shape=(10,), name='numeric_input')
x1 = layers.Dense(64, activation='relu')(numeric_input)
x1 = layers.Dropout(0.3)(x1)
x1 = layers.Dense(32, activation='relu')(x1)

# Image input branch
image_input = Input(shape=(28, 28, 1), name='image_input')
x2 = layers.Conv2D(32, (3,3), activation='relu')(image_input)
x2 = layers.MaxPooling2D((2,2))(x2)
x2 = layers.Conv2D(64, (3,3), activation='relu')(x2)
x2 = layers.MaxPooling2D((2,2))(x2)
x2 = layers.Flatten()(x2)
x2 = layers.Dropout(0.3)(x2)
x2 = layers.Dense(64, activation='relu')(x2)

# Combine branches
combined = layers.concatenate([x1, x2])

# Output 1: Regression
regression_output = layers.Dense(1, activation='linear', name='regression_output')(combined)

# Output 2: Classification (3 classes)
classification_output = layers.Dense(3, activation='softmax', name='classification_output')(combined)

# Define model
model = models.Model(inputs=[numeric_input, image_input], outputs=[regression_output, classification_output])

# Compile model with weighted losses
model.compile(optimizer='adam',
              loss={'regression_output': 'mse', 'classification_output': 'sparse_categorical_crossentropy'},
              loss_weights={'regression_output': 1.0, 'classification_output': 1.0},
              metrics={'regression_output': 'mse', 'classification_output': 'accuracy'})

# Example dummy data for demonstration
import numpy as np
X_numeric = np.random.rand(1000, 10).astype('float32')
X_image = np.random.rand(1000, 28, 28, 1).astype('float32')
y_regression = np.random.rand(1000, 1).astype('float32')
y_classification = np.random.randint(0, 3, 1000)

# Train model
history = model.fit(
    {'numeric_input': X_numeric, 'image_input': X_image},
    {'regression_output': y_regression, 'classification_output': y_classification},
    epochs=20,
    batch_size=32,
    validation_split=0.2
)

Added separate dense layers with dropout for numeric input.

Added convolutional and pooling layers with dropout for image input.

Merged processed inputs before output layers.

Used linear activation for regression output and softmax for classification output.

Compiled model with appropriate loss functions and metrics for each output.

Trained with balanced loss weights and validation split.

Results Interpretation

Before: Regression loss = 0.8, Classification accuracy = 60%

After: Regression loss = 0.35, Classification accuracy = 78%

Separately processing different input types and carefully designing outputs with suitable activations and losses helps the model learn better. Dropout reduces overfitting and balancing losses improves multi-output training.

Bonus Experiment

Try changing the loss weights to prioritize classification accuracy and observe how regression loss changes.

💡 Hint

Increase the classification loss weight to 2.0 and reduce regression loss weight to 0.5 in model.compile to focus training more on classification.

Practice

1. What is the main purpose of a multi-input model in TensorFlow?

easy

A. To accept more than one data source at the same time

B. To predict multiple outputs from a single input

C. To train faster using GPU acceleration

D. To reduce the number of layers in the model

Solution

1. Step 1: Understand multi-input models

   Multi-input models are designed to take multiple data sources as inputs simultaneously.

2. Step 2: Differentiate from multi-output models

   Multi-output models predict multiple outputs but usually from a single input source.

3. Final Answer:

   To accept more than one data source at the same time -> Option A

4. Quick Check:

   Multi-input = multiple data sources [OK]

Hint: Multi-input means many inputs, not many outputs [OK]

Common Mistakes:

• Confusing multi-input with multi-output
• Thinking multi-input reduces layers
• Assuming multi-input speeds training automatically

2. Which of the following is the correct way to define two inputs in a TensorFlow Keras model?

easy

A. inputs = tf.keras.Input(shape=(10, 5))

B. inputs = tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,))

C. inputs = [tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,))]

D. inputs = tf.keras.Input(shape=(10,)); inputs = tf.keras.Input(shape=(5,))

Solution

1. Step 1: Recall how to define multiple inputs

   Multiple inputs should be stored as a list of Input layers in Keras.

2. Step 2: Check each option

   inputs = [tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,))] correctly creates a list of two Input layers. inputs = tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,)) creates a tuple but does not assign it properly. inputs = tf.keras.Input(shape=(10, 5)) defines a single input with combined shape. inputs = tf.keras.Input(shape=(10,)); inputs = tf.keras.Input(shape=(5,)) overwrites the first input with the second.

3. Final Answer:

   inputs = [tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,))] -> Option C

4. Quick Check:

   Multiple inputs = list of Input layers [OK]

Hint: Use a list to hold multiple Input layers [OK]

Common Mistakes:

• Using a tuple instead of a list for inputs
• Overwriting inputs instead of storing both
• Combining shapes into one input incorrectly

3. What will be the output shape of the following multi-output model?

input1 = tf.keras.Input(shape=(8,))
input2 = tf.keras.Input(shape=(4,))
x1 = tf.keras.layers.Dense(5)(input1)
x2 = tf.keras.layers.Dense(3)(input2)
output1 = tf.keras.layers.Dense(2)(x1)
output2 = tf.keras.layers.Dense(1)(x2)
model = tf.keras.Model(inputs=[input1, input2], outputs=[output1, output2])
print([o.shape for o in model.outputs])

medium

A. [TensorShape([None, 2]), TensorShape([None, 1])]

B. [TensorShape([8, 2]), TensorShape([4, 1])]

C. [TensorShape([None, 5]), TensorShape([None, 3])]

D. [TensorShape([None, 8]), TensorShape([None, 4])]

Solution

1. Step 1: Trace output layers shapes

   output1 is Dense(2) applied to x1, so shape is (None, 2). output2 is Dense(1) applied to x2, so shape is (None, 1).

2. Step 2: Understand batch dimension

   TensorFlow uses None for batch size, so output shapes include None as first dimension.

3. Final Answer:

   [TensorShape([None, 2]), TensorShape([None, 1])] -> Option A

4. Quick Check:

   Output shapes match Dense layer units [OK]

Hint: Output shape = batch size null + Dense units [OK]

Common Mistakes:

• Confusing input shape with output shape
• Ignoring batch dimension null
• Mixing intermediate layer shapes with output shapes

4. Identify the error in this multi-output model definition:

input = tf.keras.Input(shape=(10,))
x = tf.keras.layers.Dense(8)(input)
output1 = tf.keras.layers.Dense(4)(x)
output2 = tf.keras.layers.Dense(3)(x)
model = tf.keras.Model(inputs=input, outputs=[output1, output2])

medium

A. inputs should be a list when there are multiple outputs

B. outputs should be a single tensor, not a list

C. inputs must be a list even if only one input exists

D. No error, the model is defined correctly

Solution

1. Step 1: Check inputs parameter

   inputs can be a single Input layer if there is only one input source.

2. Step 2: Check outputs parameter

   outputs can be a list of tensors to define multiple outputs.

3. Final Answer:

   No error, the model is defined correctly -> Option D

4. Quick Check:

   Single input + multiple outputs = valid model [OK]

Hint: Single input can be passed directly, outputs can be list [OK]

Common Mistakes:

• Thinking inputs must always be a list
• Believing outputs cannot be a list
• Assuming multiple outputs require multiple inputs

5. You want to build a model that takes two inputs: an image (shape 64x64x3) and a vector of 10 features. It should output two predictions: a 5-class classification and a single continuous value. Which is the correct way to define the model inputs and outputs?

hard

A. inputs = [tf.keras.Input(shape=(64,64,3)), tf.keras.Input(shape=(10,))]; outputs = tf.keras.layers.Dense(6)(inputs)

B. inputs = [tf.keras.Input(shape=(64,64,3)), tf.keras.Input(shape=(10,))]; outputs = [tf.keras.layers.Dense(5, activation='softmax')(x), tf.keras.layers.Dense(1)(y)]

C. inputs = tf.keras.Input(shape=(64,64,3)); outputs = [tf.keras.layers.Dense(5)(inputs), tf.keras.layers.Dense(1)(inputs)]

D. inputs = tf.keras.Input(shape=(74,)); outputs = [tf.keras.layers.Dense(5)(inputs), tf.keras.layers.Dense(1)(inputs)]

Solution

1. Step 1: Define inputs separately for image and vector

   Two inputs require two Input layers with correct shapes: (64,64,3) for image and (10,) for vector.

2. Step 2: Define outputs separately for classification and regression

   Outputs are two layers: one Dense with 5 units and softmax for classification, one Dense with 1 unit for continuous value.

3. Final Answer:

   inputs = [tf.keras.Input(shape=(64,64,3)), tf.keras.Input(shape=(10,))]; outputs = [tf.keras.layers.Dense(5, activation='softmax')(x), tf.keras.layers.Dense(1)(y)] -> Option B

4. Quick Check:

   Separate inputs and outputs for multi-input/output model [OK]

Hint: Match each input and output with separate Input and Dense layers [OK]

Common Mistakes:

• Combining inputs into one vector incorrectly
• Using single input for different data types
• Outputting combined units instead of separate outputs