Multi-input and multi-output models in TensorFlow

Introduction

Sometimes, a model needs to learn from different types of data at once or predict multiple things at the same time. Multi-input and multi-output models help with that.

When you have data from different sources, like images and text, and want to use both to make a prediction.

When you want a model to predict more than one thing, like predicting both the price and the category of a product.

When combining sensor data from multiple devices to make a decision.

When building a chatbot that takes user input and context as separate inputs and outputs both a reply and a sentiment score.

Syntax

TensorFlow

from tensorflow.keras.layers import Input, Dense, concatenate
from tensorflow.keras.models import Model

# Define inputs
input1 = Input(shape=(input1_shape,))
input2 = Input(shape=(input2_shape,))

# Process inputs
x1 = Dense(units)(input1)
x2 = Dense(units)(input2)

# Combine processed inputs
combined = concatenate([x1, x2])

# Define outputs
output1 = Dense(output1_units, activation='activation')(combined)
output2 = Dense(output2_units, activation='activation')(combined)

# Create model
model = Model(inputs=[input1, input2], outputs=[output1, output2])

Use Input() to define each input separately.

Outputs can be multiple layers, each predicting different things.

Examples

This example shows two inputs with different shapes and two outputs: one for binary classification and one for multi-class classification.

TensorFlow

from tensorflow.keras.layers import Input, Dense, concatenate
from tensorflow.keras.models import Model

input_a = Input(shape=(3,))
input_b = Input(shape=(2,))

x = Dense(4, activation='relu')(input_a)
y = Dense(4, activation='relu')(input_b)

combined = concatenate([x, y])

output1 = Dense(1, activation='sigmoid')(combined)
output2 = Dense(3, activation='softmax')(combined)

model = Model(inputs=[input_a, input_b], outputs=[output1, output2])

A simple single-input, single-output model for comparison.

TensorFlow

input1 = Input(shape=(5,))
output1 = Dense(1)(input1)

model = Model(inputs=input1, outputs=output1)

Sample Model

This program creates a model with two inputs and two outputs. It trains on random data and prints predictions and training accuracies.

TensorFlow

import numpy as np
from tensorflow.keras.layers import Input, Dense, concatenate
from tensorflow.keras.models import Model

# Define two inputs
input1 = Input(shape=(4,))
input2 = Input(shape=(3,))

# Process each input
x1 = Dense(8, activation='relu')(input1)
x2 = Dense(8, activation='relu')(input2)

# Combine processed inputs
combined = concatenate([x1, x2])

# Define two outputs
output1 = Dense(1, activation='sigmoid', name='output1')(combined)
output2 = Dense(2, activation='softmax', name='output2')(combined)

# Create model
model = Model(inputs=[input1, input2], outputs=[output1, output2])

# Compile model
model.compile(optimizer='adam',
              loss={'output1': 'binary_crossentropy', 'output2': 'categorical_crossentropy'},
              metrics={'output1': 'accuracy', 'output2': 'accuracy'})

# Generate dummy data
x1_data = np.random.random((100, 4))
x2_data = np.random.random((100, 3))
y1_data = np.random.randint(2, size=(100, 1))
y2_data = np.zeros((100, 2))
y2_data[np.arange(100), np.random.randint(2, size=100)] = 1

# Train model
history = model.fit([x1_data, x2_data], [y1_data, y2_data], epochs=3, batch_size=10, verbose=0)

# Predict on new data
preds = model.predict([x1_data[:2], x2_data[:2]])

print('Output 1 predictions:', preds[0])
print('Output 2 predictions:', preds[1])
print('Training accuracy for output1:', history.history['output1_accuracy'][-1])
print('Training accuracy for output2:', history.history['output2_accuracy'][-1])

Important Notes

Make sure input data matches the shape and order of the model inputs.

Losses and metrics can be specified separately for each output.

Naming outputs helps when compiling and training the model.

Summary

Multi-input models take more than one data source at once.

Multi-output models predict multiple things at the same time.

Use Input layers for each input and list outputs when creating the model.