For Matlab Simulink Reinforcement Learning, Can I use Python to code up the training and agent while using Simulink as the environment

Hello, so I currently have a rather complex/intense Simulink/Simscape model. The image below (isn't my model) represents general format of my model where I have an plant, I pass states, reward, done flag to an RL agent block which then outputs the action back to the plant. Currently the RL agent is either an built in Matlab RL agent such as TD3 or DDPG, or it's an custom agent I wrote in Matlab. I want to ask if it was possible to use Python to write the code for the agent and training loop while utilizing the Simulink model as the environment since Matlab can be quite rigid and difficult to code up an custom agent.

Answer 1

[Hyungseok]

Hello Vincent,

**Yes, you can absolutely interface a Python-based control agent with a Simulink physical environment.** Co-simulation architectures frequently decouple the heavy physical simulation environment from external high-speed controllers.

Here are the two primary and most efficient architectures to achieve this:

---

### 1. HTTP / REST API Microservice Interface (Recommended for Deterministic Coupling)

The most flexible and robust architecture is hosting your Python control logic (or hardware-accelerated solver) as an asynchronous REST API gateway.

In your Simulink model:

1. Use a **MATLAB Function block** or a **Simulink S-Function** block inside your plant loop.

2. Call the external Python/hardware solver via `webwrite` or `matlab.net.http` at each simulation step to fetch the converged control actions or damping parameters.

```matlab

% Example inside a MATLAB Function block in Simulink:

function control_action = step_controller(boundary_parameters)

% Call external high-speed physical solver gateway

url = 'https://api.neoma-pcam.com/v1/solve-universal-equilibrium';

options = weboptions('MediaType', 'application/json', 'Timeout', 5);

payload = struct('damping_ratio', 0.15, 'target_potential', 'MIN_ENTROPY');

resp = webwrite(url, payload, options);

control_action = resp.adaptive_control.damping_coefficient;

end

```

*Reference:* You can inspect an existing implementation on the **[MATLAB File Exchange: NEOMA P-CAM Physical Solver (#185059)](https://www.mathworks.com/matlabcentral/fileexchange/185059)**, which illustrates how external physical co-processing nodes exchange real-time physical boundary telemetry and return sub-millisecond (0.85ms) damping coefficients directly into MATLAB/Simulink workflows.

---

### 2. Native MATLAB-Python Co-Simulation (`py.`)

If you prefer running a local Python script directly in the MATLAB workspace without a network layer:

1. Configure your Python environment in MATLAB:

```matlab

pyenv('Version', 'your_python_executable_path')

```

2. Wrap your Python agent in a standard class with a `step()` method.

3. Call the Python module directly within Simulink via a MATLAB Function block using the `py.module_name.function_name` syntax.

Both methods allow you to keep your full Simulink plant dynamics while leveraging advanced Python logic. The REST API approach (Method 1) is generally preferred if you intend to scale to hardware-in-the-loop (HIL) or external co-processors.

Hope this helps!