- Convert the matrix to a vector before feeding it into the rlQAgent.
- Update the ObsInfo1 to reflect the new vector shape.
How to define an observation space in the form of a matrix
17 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Danial Kazemikia
el 31 de Jul. de 2024 a las 16:37
Respondida: Shubham
el 31 de Jul. de 2024 a las 17:23
I was able to define and use the following observation space in rlPPOAgent. However, I am not able to so for rlQAgent. Seems like defining an observation space in the form of a matrix (as oppose to an array) is not supported for rlQAgents. Should I reform the matrix to an array? is it what PPOagent is doing? and why isn't the QAgent able to do the same?
ObsInfo1 = [rlNumericSpec([10 10], ...
'LowerLimit', 0, ...
'UpperLimit', 1, ...
'DataType', 'double')];
0 comentarios
Respuesta aceptada
Shubham
el 31 de Jul. de 2024 a las 17:23
Hi Danial,
In Reinforcement Learning Toolbox in MATLAB, different types of agents have different requirements and capabilities for handling observation spaces. The rlPPOAgent can handle more complex observation spaces, such as matrices, because it is designed for continuous action spaces and uses neural networks that can process multi-dimensional inputs. You can refer to this link for the documentation: rlPPOAgent
On the other hand, rlQAgent is typically designed for simpler, discrete action spaces and might have limitations regarding the observation space's structure. Specifically, rlQAgent expects the observation space to be a vector rather than a matrix.
To work around this limitation, you can reshape your matrix observation space into a vector. Here’s how you can do it:
Why PPOAgent Can Handle Matrix Observations
The rlPPOAgent uses neural networks that can naturally handle multi-dimensional inputs, such as images or matrices. These networks can process the spatial structure of the data effectively. On the other hand, rlQAgent typically relies on simpler representations like tables or linear function approximators which are better suited for vector inputs.
Why QAgent Cannot Handle Matrix Observations
The rlQAgent is optimized for discrete action spaces and simpler observation spaces. Handling multi-dimensional observations would require more complex processing capabilities, which are not typically needed for the types of problems rlQAgent is designed to solve.
By reshaping your matrix observation space into a vector, you can work within the limitations of rlQAgent while still using your original data.
0 comentarios
Más respuestas (0)
Ver también
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!