Create policy evaluation function for RL agent

7 visualizaciones (últimos 30 días)
Juan Nunez
Juan Nunez el 23 de Sept. de 2019
Comentada: Sviatoslav Klos el 25 de Mayo de 2020
Hello,
I'm using the Reinforcement Learning Toolbox to train an agent for a particular task. Once the training is complete, I want to generate a policy evaluation function so I can use it in a simulink file. I can create the policy function using "generatePolicyFuncion" command. The matlab script that is generated is the following:
function action1 = evaluatePolicy(observation1)
%#codegen
% Reinforcement Learning Toolbox
% Generated on: 23-Sep-2019 09:27:39
action1 = localEvaluate(observation1);
end
%% Local Functions
function action1 = localEvaluate(observation1)
persistent policy
if isempty(policy)
policy = coder.loadDeepLearningNetwork('agentData.mat','policy');
end
action1 = predict(policy,observation1);
end
When I run the simulink file, the following errors pop up:
Simulink does not have enough information to determine output sizes for this block. If you think the errors below are inaccurate, try specifying types for the block inputs and/or sizes for the block outputs.
Component:MATLAB Function | Category:Coder error
Dot indexing is not supported for variables of this type.
Function 'MATLAB Function' (#171.286.341), line 13, column 11:
"coder.loadDeepLearningNetwork('agentData.mat','policy')"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment.
Function 'MATLAB Function' (#171.365.371), line 15, column 19:
"policy"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed.
Function 'MATLAB Function' (#171.140.167), line 7, column 11:
"localEvaluate(observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'test_RL/MATLAB Function'
Component:MATLAB Function | Category:Coder error
Simulink cannot determine sizes and/or types of the outputs for block 'test_RL/MATLAB Function' due to errors in the block body, or limitations of the underlying analysis. The errors might be inaccurate. Fix the indicated errors, or explicitly specify sizes and/or types for all block outputs.
Component:MATLAB Function | Category:Coder error
Dot indexing is not supported for variables of this type.
Function 'MATLAB Function' (#171.286.341), line 13, column 11:
"coder.loadDeepLearningNetwork('agentData.mat','policy')"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment.
Function 'MATLAB Function' (#171.365.371), line 15, column 19:
"policy"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed.
Function 'MATLAB Function' (#171.140.167), line 7, column 11:
"localEvaluate(observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'test_RL/MATLAB Function'
How can I solve this issue?
Thanks

Respuesta aceptada

Emmanouil Tzorakoleftherakis
Emmanouil Tzorakoleftherakis el 25 de Sept. de 2019
Editada: Emmanouil Tzorakoleftherakis el 25 de Sept. de 2019
Can you try defining the size of inputs and outputs in the MATLAB Function block? This seems to be coming up a lot in the error message
  3 comentarios
J
J el 17 de Oct. de 2019
Hi there,
Were you able to resolve this issuse? I am facing the same problem as you and haven't been able to figure it out...
SomeMatlabUser
SomeMatlabUser el 20 de Feb. de 2020
Editada: SomeMatlabUser el 20 de Feb. de 2020
Any Update? I am having the same problem. Also there is no documentation by Mathworks itself how to setup an Simulinkmodel executing the policy for code generation.

Iniciar sesión para comentar.

Más respuestas (1)

Sviatoslav Klos
Sviatoslav Klos el 3 de Mzo. de 2020
Hello.
I experienced the same issue. If you do not need code generation (only normal mode simulation is requested) I might suggest to substitute your two functions with the following code and insert it as a MATLAB User-defined fcn. Please, respond if it suits you.
function action1 = evaluatePolicy(observation1)%, policy) %#codegen
coder.extrinsic('evalin', 'predict');
action1 = single(0);
policy = evalin('base', 'policy') ;
action1 = predict(policy,observation1);
end
  2 comentarios
Juan Nunez
Juan Nunez el 15 de Mayo de 2020
Hello Sviatoslav,
Thnk your for your response.
I'm sorry for my late reply. I tried your suggestion, but the following error keeps popping up:
"Variable 'action1' has conflicting type definitions. To resolve, verify that all preceding definitions of 'action1' are consistent.
Function 'MATLAB Function' (#420.371.409), line 14, column 1:
"action1 = predict(policy,observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'Copy_of_RF_FOC6/MATLAB Function'
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'Copy_of_RF_FOC6/MATLAB Function'"
Sviatoslav Klos
Sviatoslav Klos el 25 de Mayo de 2020
I think this is your project specific error. You might check the action of the RL agent. I might suggest to test this function with some RL example from MATLAB and the try to go into your application with obtained conclusions.

Iniciar sesión para comentar.

Productos


Versión

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by