Multi-input and multi-output models in TensorFlow

Introduction

Sometimes, a model needs to learn from different types of data at once or predict multiple things at the same time. Multi-input and multi-output models help with that.

When you have data from different sources, like images and text, and want to use both to make a prediction.

When you want a model to predict more than one thing, like predicting both the price and the category of a product.

When combining sensor data from multiple devices to make a decision.

When building a chatbot that takes user input and context as separate inputs and outputs both a reply and a sentiment score.

Syntax

TensorFlow

from tensorflow.keras.layers import Input, Dense, concatenate
from tensorflow.keras.models import Model

# Define inputs
input1 = Input(shape=(input1_shape,))
input2 = Input(shape=(input2_shape,))

# Process inputs
x1 = Dense(units)(input1)
x2 = Dense(units)(input2)

# Combine processed inputs
combined = concatenate([x1, x2])

# Define outputs
output1 = Dense(output1_units, activation='activation')(combined)
output2 = Dense(output2_units, activation='activation')(combined)

# Create model
model = Model(inputs=[input1, input2], outputs=[output1, output2])

Use Input() to define each input separately.

Outputs can be multiple layers, each predicting different things.

Examples

This example shows two inputs with different shapes and two outputs: one for binary classification and one for multi-class classification.

TensorFlow

from tensorflow.keras.layers import Input, Dense, concatenate
from tensorflow.keras.models import Model

input_a = Input(shape=(3,))
input_b = Input(shape=(2,))

x = Dense(4, activation='relu')(input_a)
y = Dense(4, activation='relu')(input_b)

combined = concatenate([x, y])

output1 = Dense(1, activation='sigmoid')(combined)
output2 = Dense(3, activation='softmax')(combined)

model = Model(inputs=[input_a, input_b], outputs=[output1, output2])

A simple single-input, single-output model for comparison.

TensorFlow

input1 = Input(shape=(5,))
output1 = Dense(1)(input1)

model = Model(inputs=input1, outputs=output1)

Sample Model

This program creates a model with two inputs and two outputs. It trains on random data and prints predictions and training accuracies.

TensorFlow

import numpy as np
from tensorflow.keras.layers import Input, Dense, concatenate
from tensorflow.keras.models import Model

# Define two inputs
input1 = Input(shape=(4,))
input2 = Input(shape=(3,))

# Process each input
x1 = Dense(8, activation='relu')(input1)
x2 = Dense(8, activation='relu')(input2)

# Combine processed inputs
combined = concatenate([x1, x2])

# Define two outputs
output1 = Dense(1, activation='sigmoid', name='output1')(combined)
output2 = Dense(2, activation='softmax', name='output2')(combined)

# Create model
model = Model(inputs=[input1, input2], outputs=[output1, output2])

# Compile model
model.compile(optimizer='adam',
              loss={'output1': 'binary_crossentropy', 'output2': 'categorical_crossentropy'},
              metrics={'output1': 'accuracy', 'output2': 'accuracy'})

# Generate dummy data
x1_data = np.random.random((100, 4))
x2_data = np.random.random((100, 3))
y1_data = np.random.randint(2, size=(100, 1))
y2_data = np.zeros((100, 2))
y2_data[np.arange(100), np.random.randint(2, size=100)] = 1

# Train model
history = model.fit([x1_data, x2_data], [y1_data, y2_data], epochs=3, batch_size=10, verbose=0)

# Predict on new data
preds = model.predict([x1_data[:2], x2_data[:2]])

print('Output 1 predictions:', preds[0])
print('Output 2 predictions:', preds[1])
print('Training accuracy for output1:', history.history['output1_accuracy'][-1])
print('Training accuracy for output2:', history.history['output2_accuracy'][-1])

Important Notes

Make sure input data matches the shape and order of the model inputs.

Losses and metrics can be specified separately for each output.

Naming outputs helps when compiling and training the model.

Summary

Multi-input models take more than one data source at once.

Multi-output models predict multiple things at the same time.

Use Input layers for each input and list outputs when creating the model.

Practice

1. What is the main purpose of a multi-input model in TensorFlow?

easy

A. To accept more than one data source at the same time

B. To predict multiple outputs from a single input

C. To train faster using GPU acceleration

D. To reduce the number of layers in the model

Solution

1. Step 1: Understand multi-input models

   Multi-input models are designed to take multiple data sources as inputs simultaneously.

2. Step 2: Differentiate from multi-output models

   Multi-output models predict multiple outputs but usually from a single input source.

3. Final Answer:

   To accept more than one data source at the same time -> Option A

4. Quick Check:

   Multi-input = multiple data sources [OK]

Hint: Multi-input means many inputs, not many outputs [OK]

Common Mistakes:

• Confusing multi-input with multi-output
• Thinking multi-input reduces layers
• Assuming multi-input speeds training automatically

2. Which of the following is the correct way to define two inputs in a TensorFlow Keras model?

easy

A. inputs = tf.keras.Input(shape=(10, 5))

B. inputs = tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,))

C. inputs = [tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,))]

D. inputs = tf.keras.Input(shape=(10,)); inputs = tf.keras.Input(shape=(5,))

Solution

1. Step 1: Recall how to define multiple inputs

   Multiple inputs should be stored as a list of Input layers in Keras.

2. Step 2: Check each option

   inputs = [tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,))] correctly creates a list of two Input layers. inputs = tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,)) creates a tuple but does not assign it properly. inputs = tf.keras.Input(shape=(10, 5)) defines a single input with combined shape. inputs = tf.keras.Input(shape=(10,)); inputs = tf.keras.Input(shape=(5,)) overwrites the first input with the second.

3. Final Answer:

   inputs = [tf.keras.Input(shape=(10,)), tf.keras.Input(shape=(5,))] -> Option C

4. Quick Check:

   Multiple inputs = list of Input layers [OK]

Hint: Use a list to hold multiple Input layers [OK]

Common Mistakes:

• Using a tuple instead of a list for inputs
• Overwriting inputs instead of storing both
• Combining shapes into one input incorrectly

3. What will be the output shape of the following multi-output model?

input1 = tf.keras.Input(shape=(8,))
input2 = tf.keras.Input(shape=(4,))
x1 = tf.keras.layers.Dense(5)(input1)
x2 = tf.keras.layers.Dense(3)(input2)
output1 = tf.keras.layers.Dense(2)(x1)
output2 = tf.keras.layers.Dense(1)(x2)
model = tf.keras.Model(inputs=[input1, input2], outputs=[output1, output2])
print([o.shape for o in model.outputs])

medium

A. [TensorShape([None, 2]), TensorShape([None, 1])]

B. [TensorShape([8, 2]), TensorShape([4, 1])]

C. [TensorShape([None, 5]), TensorShape([None, 3])]

D. [TensorShape([None, 8]), TensorShape([None, 4])]

Solution

1. Step 1: Trace output layers shapes

   output1 is Dense(2) applied to x1, so shape is (None, 2). output2 is Dense(1) applied to x2, so shape is (None, 1).

2. Step 2: Understand batch dimension

   TensorFlow uses None for batch size, so output shapes include None as first dimension.

3. Final Answer:

   [TensorShape([None, 2]), TensorShape([None, 1])] -> Option A

4. Quick Check:

   Output shapes match Dense layer units [OK]

Hint: Output shape = batch size null + Dense units [OK]

Common Mistakes:

• Confusing input shape with output shape
• Ignoring batch dimension null
• Mixing intermediate layer shapes with output shapes

4. Identify the error in this multi-output model definition:

input = tf.keras.Input(shape=(10,))
x = tf.keras.layers.Dense(8)(input)
output1 = tf.keras.layers.Dense(4)(x)
output2 = tf.keras.layers.Dense(3)(x)
model = tf.keras.Model(inputs=input, outputs=[output1, output2])

medium

A. inputs should be a list when there are multiple outputs

B. outputs should be a single tensor, not a list

C. inputs must be a list even if only one input exists

D. No error, the model is defined correctly

Solution

1. Step 1: Check inputs parameter

   inputs can be a single Input layer if there is only one input source.

2. Step 2: Check outputs parameter

   outputs can be a list of tensors to define multiple outputs.

3. Final Answer:

   No error, the model is defined correctly -> Option D

4. Quick Check:

   Single input + multiple outputs = valid model [OK]

Hint: Single input can be passed directly, outputs can be list [OK]

Common Mistakes:

• Thinking inputs must always be a list
• Believing outputs cannot be a list
• Assuming multiple outputs require multiple inputs

5. You want to build a model that takes two inputs: an image (shape 64x64x3) and a vector of 10 features. It should output two predictions: a 5-class classification and a single continuous value. Which is the correct way to define the model inputs and outputs?

hard

A. inputs = [tf.keras.Input(shape=(64,64,3)), tf.keras.Input(shape=(10,))]; outputs = tf.keras.layers.Dense(6)(inputs)

B. inputs = [tf.keras.Input(shape=(64,64,3)), tf.keras.Input(shape=(10,))]; outputs = [tf.keras.layers.Dense(5, activation='softmax')(x), tf.keras.layers.Dense(1)(y)]

C. inputs = tf.keras.Input(shape=(64,64,3)); outputs = [tf.keras.layers.Dense(5)(inputs), tf.keras.layers.Dense(1)(inputs)]

D. inputs = tf.keras.Input(shape=(74,)); outputs = [tf.keras.layers.Dense(5)(inputs), tf.keras.layers.Dense(1)(inputs)]

Solution

1. Step 1: Define inputs separately for image and vector

   Two inputs require two Input layers with correct shapes: (64,64,3) for image and (10,) for vector.

2. Step 2: Define outputs separately for classification and regression

   Outputs are two layers: one Dense with 5 units and softmax for classification, one Dense with 1 unit for continuous value.

3. Final Answer:

   inputs = [tf.keras.Input(shape=(64,64,3)), tf.keras.Input(shape=(10,))]; outputs = [tf.keras.layers.Dense(5, activation='softmax')(x), tf.keras.layers.Dense(1)(y)] -> Option B

4. Quick Check:

   Separate inputs and outputs for multi-input/output model [OK]

Hint: Match each input and output with separate Input and Dense layers [OK]

Common Mistakes:

• Combining inputs into one vector incorrectly
• Using single input for different data types
• Outputting combined units instead of separate outputs