Using Simulink with Python
Engineers use MATLAB® and Simulink® with other programming languages, for example C/C++ or Python®, to develop algorithm components for their increasingly complex projects. Integrating these components, which are developed in different environments, together for system-level simulation is becoming a key step of the project’s success.
In this presentation, see a demonstration of major workflows using Python with Simulink. Learn how to use Python Importer and other methods to bring Python code (including AI models and data processing pipelines) easily into Simulink. In addition, discover how you can simulate your Simulink model from a Python-based environment through the MATLAB engine or compile your Simulink model as a Python package for deployment.
Published: 3 May 2023
[AUDIO LOGO]
Hello, everyone. Thank you for joining us today for MATLAB EXPO. And I'm here with Wenwu. Wenwu, if you want to introduce yourself shortly.
Yeah. Hi, everyone. My name is Wenwu Li. I'm the Product Manager at The MathWorks. It's great over here.
So today, we will be talking to you about Simulink with Python. And my name is Yann. You probably have heard me speaking in the past about MATLAB with Python. But today, I've decided to invite Wenwu to talk to us about a more specific topic that is Simulink with Python. And before we dive into the subject, Wenwu, why don't you share with us a few key takeaways?
Sure. After this presentation, the key takeaway I hope you can bring home is that Simulink is open simulation platform. It supports multiple ways to interoperate with Python. You can bring in Python functions as library blocks for code execution.
For your deeper learning models, you can integrate them with the Simulink for both simulation and code generation. If your working environment is a Python, you can simulate a Simulink model directly in Python. Or you can export your model as a Python package for deployment.
So all of this sounds good, but you may ask yourself, why? Why would I want to be using Python and Simulink together? And learn a little more about what you will tell us. Three, four reasons that we often encounter is first just a simple need to integrate code from a colleague.
There are some subjects related to AI workflow-- how to facilitate the development of such a workflow. Some need for functionalities that may be only available in MATLAB or maybe in Python. And in this case here, the killer app, let's say, is Simulink. And sometimes also to leverage the work from the community, you can either find a model online or maybe some Python scripting that you may want to be reusing.
So for this, we will just use four typical scenarios that we encounter on a daily basis. And in this setting, Wenwu will we be playing the role of the Simulink user and I will be playing the role of the Python user. And in each of those projects, we will see how we can work together to deliver on that.
So let's start with scenario number one. In this case, I'm an algorithm developer. I'm using Python. And I developed an image processing and computer vision algorithm.
And I want to work with you, Wenwu. You're a system engineer. And you want to integrate my algorithm inside of your complete system simulation. So can you tell me a little bit about it?
Sure. Actually, in 23a, we shipped a new feature called the Python Importer. It can help you to bring your Python function in the Simulink as a library block easily. First of all, Python Importer provides a graphical wizard for step-by-step guidance. During the importing process, you don't need to have a lot of menu code-- or maybe just the minimal code is needed. With this generated library block, you can conveniently reuse your design and simulate your Python algorithm directly in Simulink.
So here, I have a short video just to show you how to integrate human detection algorithms in Python. And let's say this is the Python algorithm. I got it from. Now I need to bring this into Simulink. So from the command line, you can open the Python Importer wizard.
Once Python Importer is open, now we can start the importing process. The first step is for you to specify the Simulink library file name. Let's say this is HumanDetection. Then you can specify where to put this library file.
OK. After this, it will ask you which files you want to import in the Simulink. So I'll select detecthuman. That's the Python file I showed you before. Then after this step, Simulink will analyze the Python algorithm automatically and will capture the function here.
Then at this step, you will have the opportunity to specify the ports, including the data type, the size of your data-- and also you can specify the name, input, output, including parameters. Here, because of the input and output images. So it's a 3D size.
After that, then you can see your library is being created. Now as a quick unit test, let's bring this library into Simulink for simulation. As you can see, it's pretty much doing the work as expected.
All right. So that's for 23a. Now, if you're using an older release, you may wonder what options you have. Well, for older releases, you can still do it. It only takes a little more minor steps.
First, you need to select either MATLAB function block or MATLAB system block to write your MATLAB code. Then your MATLAB code-- you can access the Python libraries directly by adding the py prefix to the Python name. This is because MATLAB provides an embedded interface for Python. OK, so that's the first scenario.
Thank you. So it's quite interesting how you can co-simulate Python with Simulink. But obviously, the question you may have is what about code generation? And so in this second scenario, I'll still be playing the role of a data scientist. But in this case, I'll be using TensorFlow or PyTorch in order to develop deep learning models that could be, for instance, a battery instead of charge estimation.
And in this scenario, you'll be leveraging the pretraining deep learning models that I've developed and integrated inside of Simulink for both simulation-- but also, more importantly, for generation. Before we see how to do that, there are two options. The first one is imparting the deep learning model in MATLAB directly with the MATLAB Deep Learning Toolbox. And the second one is to simulate and generate code through the use of the TensorFlow Lite model.
The plus and delta for each of those approaches are for the input directly into MATLAB, the fact that it generates multiple platform for co-generation. Those are library-free C and C++ code, some optimized code for Intel and ARM processor, and CUDA code for GPUs. The delta is, in this case, that the input process can sometimes be painful. And there's a bit of a need of custom code and manual validation testing.
On the other hand, for you to leverage TensorFlow Lite, you only require a simple Python line of code to compile the model. So the preparation is fairly simple. But on the downside, it requires you to have the TensorFlow Lite interpreter and the library built for the target hardware. So it's really limiting to Windows and Linux target for now-- so very complementary, those two approaches.
So here, I'm going to introduce the first approach to convert your Python-based deep learning model into a MATLAB deep learning model. You can convert models from TensorFlow, PyTorch, and ONNX into MATLAB. Once the model is converted in MATLAB, you can use the deep learning blocks in Simulink to print those models in the Simulink for both simulation and code generation.
So here, I'm going to show a demo of how to integrate a TensorFlow model for battery state of charge estimation. Battery state of charge estimation is essential for developing electrical vehicles. There are several ways to do it, and using deep learning is one way to do this.
Let's assume you already have a TensorFlow model set up and pretrained. So you can use the input tensorflow network to bring this into MATLAB. And the next step is very convincing-- you can actually visualize the imported network in MATLAB. This is not necessary, but it can help you to understand what the imported network looks like-- for example, we have nine layers here. Everything looks good.
Then you can save your model for re-use. So here, you open Simulink, you can use the deep learning block to bring this [INAUDIBLE] file into Simulink for simulation and also code generation. As you can see here, the deep learning prediction follows the True_SOC very well.
So I think it's a great approach for you if you're a MATLAB in Simulink user. But now, if in my scenario, I'm more of a Python user, one other complementary approach would be to iterate through the use of what is called TensorFlow Lite. And as you can see in this example, you have a block in which you can directly integrate the code to the TensorFlow Lite model. And when generating the equivalent of this code-- well, what you will get is here, as you can see on the right-hand side. Some of the code that is generated that wraps this code to the TFLite runtime.
So that was, I think, a good example that includes code generation. Now let's flip for scenario number three the order in which we look at this collaboration. And let's prioritize your Simulink model with an example of a dynamic system like your vehicle suspension.
So what I want to do is essentially call this Simulink model that you share with me in order to generate simulations. And in order to do that, the first approach-- for instance, for automated testing-- would be to call Simulink from Python. So maybe, Wenwu, you can share with us an example like this.
Definitely. Before we show the example, let's have a basic background setup here. To simulate a Simulink model directly in Python, you would need to use MATLAB engine API for Python. Basically, it is a library you need to install in your Python. Then you can start the MATLAB engine in Python. It works as a client process to facilitate the data exchange between your Python program and your MATLAB and Simulink.
This way, you can call Simulink for simulation. And that is very similar to how you do it natively in MATLAB. So here, I have a little code snipped, just to show you how you can start the MATLAB engine, and also how you can run the Simulink simulation in Python.
Before I show the demo, I would also like to add a few comments to use a MATLAB engine API. With MATLAB engine API, you can create and terminate a MATLAB session. You can also put variables from Python into MATLAB workspace. You can also get variables from MATLAB work space to Python. It also provides flexible Simulink simulation capabilities, including changing non-tunable parameters and running simulation in normal mode. So it's really convenient.
So here, I'm going to show you a demo-- how to simulate a road suspension model in Python. So here, this is a classic Simulink model. So the first step is to use a MATLAB function to wrap the simulation. To do that, you can use the simulation input object.
It is very convenient to set up a simulation-- for example, if you want to update the model parameters or even the block parameters or parameter variables. You can alter that easily through a simulation input object. And of course, you need to reuse the same command to run the simulation. Then here, this is how you're going to call the simulation in Python.
The first step is to ensure the MATLAB engine API for Python has been installed correctly. Then you can start the MATLAB engine library in Python. And here, I'm going to show two simulations. The first is just to specify a stop time, but the others are default.
Then the second is actually I'm going to change the vehicle-- the body mass to 5,000 kilograms. And also ask MATLAB to generate the visualization for us. So here, this is the MATLAB plotting. But everything actually is implemented from the Python side, so that's interesting.
So final scenario-- we would follow with the same example of a vehicle suspension system. And this time, I not only want to be coding your model, but I also want to be able to deploy it into a Python-based production environment. And for us to do that in this case, we will be leveraging the generation of a Python package that encapsulates your Simulink model and simulation. So can you share with us how it's working for this scenario?
Definitely. Yeah, this is a pretty popular scenario right now. For deployment, you can use the Simulink Compiler and Matlab Compiler SDK to convert your simulation code into a Python package. Then in Python, you can import and use this package similar to other Python packages you are using. In this case, we also need to use MATLAB Runtime with the generated Python package.
MATLAB Runtime is a standard set of shared libraries that enable the execution of MATLAB and Simulink files on computers where MATLAB and Simulink is not available. To use the deployed simulation, the Python code here is pretty much the same as in the previous interactive method. But instead of starting a MATLAB session, we are initializing the Runtime process. So from the code snippet here, when we say initialize, we are not initializing the MATLAB. We are initializing the Runtime instead.
OK. So here, I'm going to show you this suspension system demo again. This time, we are calling the compiled Simulink model. All right. So the model is the same. And also, you need to use the MATLAB Wrap function to wrap the simulation.
Whole other steps pretty much are the same. The only difference here is you need to configure your model for deployment. That's because not all Simulink models are deployable. So you need to still make the specific steps here for deployment.
Then the next step, you can compile the wrap function as a Python package. Basically, there's two ways to do that. One is to use a script, basically. That's what we're doing here.
Then the second way to show here, you can use a graphic UI to compile your MATLAB code as a Python package. Then after that, let's say if you have the compiled package and you have already installed that package correctly in Python.
As I said before, the first step is to initialize the MATLAB Runtime. That's pretty much what we're doing here. Then the next step, we are doing very similar things. The first is to specify stop time and using the default parameters. And the second simulation, we actually change the body mass to 5,000. And here, we run everything in Python. And we get the visualization from Python as well.
Before I wrap up this section, I would like to call out that in addition to compiling a Simulink simulation as a Python package, there are several other ways that you can generate a core compile Simulink model in Python. The first is you can pack the Simulink model as a function to mock up a unit or FMU. Then you can call this FMU from Python using third party libraries such as FMPy.
Then the second alternative is to package the Simulink model as a simulation service API using MATLAB Production Server. This is a good way for scalable applications. Then the last alternative is to generate C/C++ code or shared library for the Simulink model. Then you can code the generated code-- the C types or related wrappers.
Thank you, Wenwu. That sounds good. So if we were to summarize what we've learned today, you showed us and demonstrated how Simulink is an open simulation platform that supports several ways to interoperate with Python. We've seen four main ways.
First, bringing your Python code into Simulink as a library, but only working in co-simulation. The second approach, more specific to deep learning models in TensorFlow and PyTorch was enabling both simulation and code generation inside of Simulink. And on the second part, we saw how to simulate the Simulink model directly from Python, and also how to export this model as a Python package.
[AUDIO LOGO]