Using getValue in matlab fcn block in simulink
3 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
I've trained a DDPGAgent and I am trying to test the critic in simulink. I tried using matlab fcn block. However, the error showed up as figure below when I ran it. Is there any method to get value from critic network in simulink?
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/331304/image.png)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/331307/image.png)
0 comentarios
Respuestas (2)
Emmanouil Tzorakoleftherakis
el 13 de Jul. de 2020
Editada: Emmanouil Tzorakoleftherakis
el 6 de Mzo. de 2021
Hi Sam,
Before R2020a, the easiest way to bring the critic in Simulink without using the Agent block is to call generatePolicyFunction to generate a script that does inference, and then use a MATLAB Fcn block to call the generated script. You may need to add 'coder.extrinsic' statements at the begining of the script for things that do not support code generation, but that should work.
In R2020b, Deep Learning Toolbox ships a couple of blocks that allow you to bring deep neural networks into Simulink. This would be a faster way to do the same.
To do inference on the critic in Simulink before R2020b, create a function that does inference like the following:
function q = evaluateCritic(observation1)
q = localEvaluate(observation1);
end
%% Local Functions
function q = localEvaluate(observation1)
persistent policy
if isempty(policy)
policy = coder.loadDeepLearningNetwork('agentData.mat','policy');
end
q = predict(policy,observation1);
end
and in the MATLAB Fcn block in Simulink put the following
function q = MATLABFcn(observation1)
coder.extrinsic('evaluateCritic')
q=single(zeros(1,2));%dimensions should match number of actions
q = evaluateCritic(observation1);
end
Note that it's preferrable to use the critic architecture that has a single input channel and outputs multiple values based on the number of actions. I tested the above and works in R2020a.
Hope that helps
4 comentarios
Emmanouil Tzorakoleftherakis
el 15 de Jul. de 2020
Sorry, to be clear I was referring to cases where you have discrete action spaces like DQN (this link shows how to do this).
Another option to do this using 'getValue' is to create the following inference function
function q = evaluateCritic(observation, action)
persistent critic
if isempty(critic)
temp = load('SimulinkPendulumDDPG.mat','agent') %load your agent;
critic = getCritic(temp.agent); %extract critic
end
q = getValue(critic, {observation}, {action});
end
and put the following in the MATLAB Fcn block
function q = evaluateCritic(observation, action)
coder.extrinsic('evaluateCritic');
q = single(0);%initialize output
q = evaluateCritic(observation, action);
end
Also tested in R2020a. Hope that helps
Sam Chen
el 20 de Jul. de 2020
Editada: Sam Chen
el 20 de Jul. de 2020
Sorry for the late reply. It works very well in my case. I really appreciate your detailed and clear explanation. I remember that there is a button for accepted answer but I can't find it. So I voted for your answer insteadly.
0 comentarios
Ver también
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!