How to load trained R-CNN model to SIMULINK

Below is the MATLAB code that I used to train my R-CNN Object Detector

   rcnn = trainRCNNObjectDetector(ROI, cifar10Net, options, ...
    'NegativeOverlapRange', [0 0.3], 'PositiveOverlapRange',[0.5 1])

    save('C:\Users\PC1\Desktop\rcnn_Model_2.mat','rcnn');

Now my problem is the blocks like 'from File' in SIMULINK can not take any MAT file that are not timeseries object.

Below in the data format that I saved using the codes above,

Name      Size             Bytes  Class                 Attributes

rcnn      1x1             482631  rcnnObjectDetector

Below is the properties of the model after being loaded,

rcnn = 

rcnnObjectDetector with properties:

              Network: [1×1 SeriesNetwork]
           ClassNames: {'Left'  'Stop'  'Go'  'Right'  'Background'}
    RegionProposalFcn: @rcnnObjectDetector.proposeRegions

So, is that any possible way to import the trained network to SIMULINK?

Answer 1

[David Willingham]

As of R2021b, you can simulate and generate code for deep learning object detectors in Simulink®. The Analysis & Enhancement block library from Computer Vision Toolbox™ now includes the Deep Learning Object Detector (Computer Vision Toolbox) block. This block predicts bounding boxes, class labels, and scores for the input image data by using a specified trained object detector. This block enables you to load a pretrained object detector into the Simulink model from a MAT file or a MATLAB function.

For more information about working with the Deep Learning Object Detector block, see Lane and Vehicle Detection in Simulink Using Deep Learning. To learn more about using deep learning with Simulink, see Deep Learning with Simulink.

[Muhammad Faisal Khalid]

whats about the LSTM neural network. i have lstm model (time series equence) which i want to load in my simulink model.

how can i proceed with it.

i have seen the lane and vehicle detection in simulink by deep learning. when i assign my model in mat file, it gives error that there is no time series data.

[David Willingham]

You can use the Stateful predict, or Stateful classify to for using a trained LSTM with Simulink

Here are some links:

Predict and Update Network State in Simulink

Classify and Update Network State in Simulink

Answer 2

[Walter Roberson]

You could possibly create a Simulink bus object with custom signals. However, I find no evidence that Simulink has any cnn related blocks, so you would have to be doing everything opaquely though a MATLAB Function Block or Level 2 MATLAB function. I suspect that it would not be possible to use it with any accelation mode other than "normal" or "accelerated" (using coder.extrinsic), not rapid accelation or code generation. I have not cross-checked that limitation though.

[Khai Hong Chin]

I did tried to use MATLAB Function block in order to load the trained model with codes shown below.

 function y = fcn
 y=load('C:\Users\Acer\Desktop\Progress 18-11\rcnn_Model_2.mat');
 y=y.rcnn
 end

However, the diagnostic report showed the following error

 Found unsupported class for variable using function 'load'.
 MATLAB class 'rcnnObjectDetector' found at 'y.rcnn' is unsupported.

[Walter Roberson]

Nothing in the Neural Network Toolbox is supported by Simulink Coder http://www.mathworks.com/help/simulink/ug/functions-supported-for-code-generation--alphabetical-list.html; not much of the Neural Network Toolbox is supported for MATLAB Compiler; https://www.mathworks.com/products/compiler/supported/compiler_support.html. For MATLAB Coder https://www.mathworks.com/help/coder/ug/functions-supported-for-code-generation--categorical-list.html#buimyq3 see https://www.mathworks.com/help/nnet/ug/deploy-neural-network-functions-and-objects.html which will point you to genFunction which will generate code to deploy an already-trained Neural Network.

Answer 3

[De M]

Me too

Answer 4

[michael bronnmann]

Any news on this topic? I have a similar topic to use a trained model in Simulink. Has anyone an idea how to export the net to e.g. c++ for use within a sfunction?

Answer 5

[Jonathan Akorli]

Try using a MATLAB Interpreted function. You may have to use mapminmax to map the function block but its worth a try

[michael bronnmann]

Thanks for your answer. In the meanwhile I took a MATLAB Fcn block for "Shallow Neural Networks". They can be exported to either a m-function or a Simulink model.

Unfortunately there's no conversion possible from a Deep Neural Network.

Answer 6

[Cesar García Echeverry]

Did you solved the problem? i want to load a .net from Matlab to Simulink, but i'm having the same problem.

Regrads

[Muhammad Faisal Khalid]

did u find the solution as i am also facing same problem