Start Q-Learning simulation with a predefine Q-Table

I'm working on a project in were I need to start a simulation with a predefined Q-Table, this is, I have a matrix with the same size of states and actions filled with scalar values. Problem is when matlab creates a rlTable, with this command the table initializes with the matrix with all values are 0s.

What I want to know is if it posible to create a rlTable from a previously created/store matrix and initialize the rlTable with the values of the predefine matrix. Is this a possibility?

I want to initialize the rlTable with the following matrix

Answer 1

[Harsha Vardhan]

Hi,

I understand that you want to initialize the 'rlTable' with the values of the predefined matrix.

Assuming you have 243 states and 5 actions, this can be done as follows.

obsInfo = rlFiniteSetSpec([1:243]);
actInfo = rlFiniteSetSpec([1:5]);
qTable = rlTable(obsInfo,actInfo); %This step initializes the Q-table to all 0s.

%Here, I am generating a random matrix for testing.
% Instead, you can use your matrix directly in the next line of code
mean_QTables = rand(243,5); 

% This will assign the scalar values 
% present in 'mean_QTables' matrix to the rlTable variable 'qTable'
qTable.Table = mean_QTables; 

For more information, please refer to these resources.

1. Q-Value function approximator object for reinforcement learning agents - MATLAB: https://www.mathworks.com/help/reinforcement-learning/ref/rl.function.rlqvaluefunction.html#mw_d745cee1-79f5-43b5-9ca0-274a06be0272
2. Value table or Q table - MATLAB: https://www.mathworks.com/help/reinforcement-learning/ref/rltable.html
3. Create specifications object for a finite-set action or observation channel - MATLAB: https://www.mathworks.com/help/reinforcement-learning/ref/rl.util.rlfinitesetspec.html

Hope this helps in resolving your query!

[Ander Alberro]

Hi sir,

It perfectly worked. I could properly created the the Q-Table with the wanted dimensions no problem but I was having trouble in initializing the rlTable with a saved Q.matrix after training to use it in the validation step.

THANK YOU VERY MUCH!