Reinforcement Learning Toolbox: Discount factor issue
13 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Hi,
I am trying to apply some RL algorithms in the RL toolbox such as ,the actor-critic algorithm, to a problem where the rewards for each step in an episode is discounted, though in the training manager window I see the episode reward as the cumulative reward rather than the discounted sum of rewards. I wonder if this is a bug as these seems confusing .
Thanks,
0 comentarios
Respuestas (3)
Ajay Pattassery
el 26 de Ag. de 2019
Editada: Ajay Pattassery
el 26 de Ag. de 2019
In the Episode Manager you could view the discounted sum of rewards for each episode named as Episode Reward. This should be the discounted sum of rewards over the time steps if you have set rlACAgentOptions to a discount factor as below.
opt = rlACAgentOptions('DiscountFactor',0.95)
If you are observing the reward on each episode is not the discounted sum of rewards, revert with env, critic, actor, trainOpts to reproduce the issue (Or the code you have used).
0 comentarios
EBRAHIM ALEBRAHIM
el 26 de Ag. de 2019
1 comentario
Ajay Pattassery
el 29 de Ag. de 2019
Hello,
I have tried an Actor-Critic example by following the model given in the link. I can see the effect of the discount factor in the following example.
EBRAHIM ALEBRAHIM
el 29 de Ag. de 2019
1 comentario
Ajay Pattassery
el 5 de Sept. de 2019
The episode manager is showing the undiscounted cumulative reward from the environment. The discount factor, however, has an impact on training and hence the learned policy. You can observe the same by finding the average reward over a reasonable number of episodes with a discount factor closer to zero and with closer to one.
Ver también
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!