How can i scale the action of DDPG agent in Reinforcement Learning?

Hello everyone ,

I have an enveriment in simulink whose action should be vary between 0-1. Althought i am using sigmoidLayer at the final layer of the actor, in some episode the action exceed the boundry of 0-1 in the trainig process.

So, how can i fix it?

Maybe the "scailingLayer" help for it, but i don't know all values of the action in whole trainig process. So, the value of the bias and scale in "scailingLayer" command is unknown.

Is there any solution ?

Thax for any help.

Answer 1

[Sam Chak]

Hi @awcii

Sound like a constraint to me. This example shows how to train the RL agent for Lane Keeping Assist, where the front steering angle (agent) is only capable of being steered from –15° to 15°.

https://www.mathworks.com/help/slcontrol/ug/train-rl-agent-for-lane-keep-assist-with-constraint-enforcement.html

Hope it helps!

Answer 2

[Emmanouil Tzorakoleftherakis]

DDPG training works by adding noise on top of the actor output to promote exploration. In that case you may see constraint violations, so you can adjust the noise options under ddpg training options (specifically mean and variance) or you can handle the violation on the environment side by adding saturation blocks.