How to Use Reinforcement Learning in Simulink: Step-by-Step Guide

To use reinforcement learning in Simulink, create an environment model in Simulink, define an RL agent using MATLAB's Reinforcement Learning Toolbox, and train the agent by connecting it to the Simulink model. Use the rlSimulinkEnv function to interface the Simulink model with the RL agent for training and simulation.

Syntax

Here is the basic syntax to set up reinforcement learning in Simulink:

• env = rlSimulinkEnv('modelName', 'agentBlock', obsInfo, actInfo); creates the RL environment from your Simulink model.
• agent = rlAgent(observationInfo, actionInfo, options); defines the RL agent with observation and action specifications.
• trainStats = train(agent, env, trainingOptions); trains the agent using the environment.

Each part connects your Simulink model to the RL agent for training and simulation.

matlab

env = rlSimulinkEnv('mySimulinkModel', 'RL Agent Block', obsInfo, actInfo);
agent = rlDQNAgent(obsInfo, actInfo, agentOptions);
trainStats = train(agent, env, trainingOptions);

Example

This example shows how to create a simple reinforcement learning environment in Simulink, define a DQN agent, and train it.

matlab

% Define observation and action spaces
obsInfo = rlNumericSpec([1 1]);
actInfo = rlFiniteSetSpec([0 1]);

% Create Simulink environment
env = rlSimulinkEnv('simpleRLModel','RL Agent Block',obsInfo,actInfo);

% Create DQN agent options
agentOpts = rlDQNAgentOptions('UseDoubleDQN',false);

% Create DQN agent
agent = rlDQNAgent(obsInfo, actInfo, agentOpts);

% Training options
trainOpts = rlTrainingOptions('MaxEpisodes',100,'MaxStepsPerEpisode',200);

% Train the agent
trainingStats = train(agent, env, trainOpts);

Output

Training started...
Episode 1/100: Reward = ...
...
Training completed.

Common Pitfalls

Common mistakes when using reinforcement learning in Simulink include:

• Not properly defining observation and action specifications, causing errors during training.
• Incorrectly connecting the RL Agent block inside the Simulink model.
• Using incompatible agent types for the problem (e.g., discrete vs continuous actions).
• Ignoring simulation step sizes and sample times, which can affect training stability.

Always verify your Simulink model runs correctly before training and check the agent and environment compatibility.

matlab

%% Wrong way: Missing observation info
% env = rlSimulinkEnv('model','AgentBlock'); % Missing obsInfo and actInfo causes error

%% Right way:
obsInfo = rlNumericSpec([1 1]);
actInfo = rlFiniteSetSpec([0 1]);
env = rlSimulinkEnv('model','AgentBlock',obsInfo,actInfo);

Quick Reference

Summary tips for reinforcement learning in Simulink:

• Use rlSimulinkEnv to link your Simulink model with the RL agent.
• Define clear observation and action spaces matching your model signals.
• Choose the right agent type (DQN, PPO, etc.) based on action space.
• Set training options carefully to balance speed and learning quality.
• Test your Simulink model standalone before training.

Key Takeaways

Use rlSimulinkEnv to connect Simulink models with reinforcement learning agents.

Define observation and action specifications clearly to avoid training errors.

Choose an agent type that matches your action space (discrete or continuous).

Verify your Simulink model runs correctly before starting training.

Adjust training options to improve learning performance and stability.