RL Stop training criteria

I have an simulink RL environment that I would like to train in real-time (with a signal from a DAQ). I placed the agent in a triggered subsystem that is triggered by non-periodic events from the DAQ (example, the agent is triggered at t=0.95, t=2.01, t=2.98 etc). I would like the agent to train for 40 minutes at a time, but to keep training the agent over multiple days.

I have noticed that the agent continues training for a given episode after it reaches the stopping critera. For example, say that I my agent to train for 3 episodes with a maximum of 10 steps per episode. If I set my stopTrainingCriteria to 5 steps, the agent will continue to train until the episode is over.

I find that this same behavior occurs with the save training criteria. If I set the save agent criteria to 5 steps, when I look at the folder where the saved agents are saved, I will see only 3 saved agents - 1 for each episode, instead of 10 saved agents.

Answer 1

[Emmanouil Tzorakoleftherakis]

I believe that for event-based training, you need to adjust your stopping/saving criteria accordingly. For example the agent will only take a step if an event is triggered. So if you set your stopping criteria to 5 steps and the training episode does not terminate prematurely, that probably means that you have less than 5 events happening in that time frame. Same thing for saving criteria.