Problem with RL agent block

I have this simulink model which mainly utilizes functions blocks to process signal. The observation input for the RL agent is a scalar. obsinfo uses rlNumericSpec and ainfo uses rlFiniteSetSpec given its discrete property. I intend to validate the environment generated via the function env = rlSimulinkenv(mdl,agentblk,obsinfo,ainfo). To validate, I uses the function validateEnvironment(env). This generates an error.

The error states:

Error using rl.env.SimulinkEnvWithAgent/validateEnvironment. An error occurred while running the simulation for the model 'simulink2' with the following RL agent blocks: simulink2/RL agent

Model 'simulink2' has underspecified signal dimensions and Simulink has assigned these dimensions by default. To prevent ambiguities, consider speficying these dimensions in the block dialog box. Simulink has assigned these signal dimensions:

• Block 'simulink2/RL Agent/last_observation' 'Output Port 1' has been set to [1]`

Before, I encounter 3 bullets with the same error. The two were pertaining to the same error regarding the specification of signal dimensions for last_action and external_action. This error were solved by enabling the RL agent parameter for this and assigning the appropriate signal values.

Now for the last_observation, I do not see how to access this parameter to assign the appropriate dimension in the RL agent block dialog box. What would be the way to let the model know that the dimension for the last_observation is indeed [1]? Is there a way to access this parameter in the block dialog box and where?

Answer 1

[Emmanouil Tzorakoleftherakis]

You can use a delay block for the last observation and set the initial value of the delay in the block dialog. That should resolve the dimension issue for this signal.

[Ethan Serapio]

I have solved the problem by adding a delay block in the model. Also, I have found in several forums that this is not an error but rather a warning although it appears as an error in the matlab window. The prompt may appear as an error given that it has a red font but in the default settings of the configuration in simulink, it is set as a warning. By this, the prompt can be disregarded by setting it as 'none' instead of 'warning'. The train function stops even for warnings thus the solution.

This option may not be practical for other models if the assumption of matlab with the dimension is incorrect, otherwise it is valid. This is also the solution for removing the matlab error regarding last_action and for the other one which I forgot which goes along with the error I've posted.

I have my RL model up and running. Thank you for your response.