Seeking assistance creating a frequency matrix

Hello,

I was hoping that somebody might be able to help me out in creating a frequency matrix?

I normally use excel for statistical modelling but this particular problem takes hours to execute using spreadsheets.

I have created a variables called 'data' and it contains numbers ranging from 0-828.

I was hoping to create a frequency matrix of the number of times one number follows another number. Or put a different way, the number of times a number transitions to another number.

Here is a simple example with numbers ranging from 1-5.

For example, the cell in red, the number 3 is followed by 2 twice, or transitions to 2 twice.

I'd appreciate any help that is offered.

Kind Regards

John

Answer 1

[Walter Roberson]

dcol = data(:);  %need column vector form
freqmatrix = accumarray( [dcol(1:end-1) + 1, dcol(2:end) + 1], 1 );

This would create a matrix up to 829 x 829. Any one position, (R,C) in it, would indicate a transition from (R-1) to (C-1). The offset of 1 is needed because you have 0 values and subscripts cannot be 0.

If the number of unique values is considerably smaller than 829, then it would be possible to make the matrix much smaller, at the expense of making it less natural to read.

[John]

Hello Walter,

Thank you for your help. I will test this out and let you know how I get on.

Kind Regards

[John]

Thanks you Walter, this does exactly what I wanted. Have a good day.

[Roger Stafford]

If your data values are not positive integers or are not closely-packed near zero, you can do a 'unique' call first. To use Walter's nomenclature,

 [ud,~,nd] = unique(data(:));
 N = length(nd);
 freqmatrix = accumarray([nd(1:N-1),nd(2:N)],1,[N,N]);

The rows and columns in N x N 'freqmatrix' would then correspond to values in 'ud'.

Answer 2

[Image Analyst]

John, though you don't know it (yet), what you are describing is called the "gray level co-occurence matrix". This is done by graycomatrix() in the Image Processing Toolbox.

[Walter Roberson]

I thought that was two-way, that if A was next to B then B would be considered next to A ? That is not the case for transition matrices, though.

[Image Analyst]

Can be, but not in the way MATLAB does it. Here, run this code:

matrix = magic(3)
glcm = graycomatrix(matrix, 'GrayLimits', [1 9], 'offset', [0 1], 'NumLevels', 9)

You'll see this:

matrix =
     8     1     6
     3     5     7
     4     9     2
glcm =
     0     0     0     0     0     1     0     0     0
     0     0     0     0     0     0     0     0     0
     0     0     0     0     1     0     0     0     0
     0     0     0     0     0     0     0     0     1
     0     0     0     0     0     0     1     0     0
     0     0     0     0     0     0     0     0     0
     0     0     0     0     0     0     0     0     0
     1     0     0     0     0     0     0     0     0
     0     1     0     0     0     0     0     0     0

The way to read this is that the number in the column is how many times the column number follows the row number. So you'd read this array as follows (going down row by row):

6 follows a 1 one time.
nothing follows a 2.
5 follows a 3 one time.
9 follows a 4 one time.
7 follows a 5 one time.
nothing follows a 6.
nothing follows a 7.
1 follows a 8 one time.
2 follows a 9 one time.

Using offset you can specify any direction you want. So to have a 2 (like you said) you'd have to do both directions and add the glcm matrices together.

[Walter Roberson]

Interesting, I would not have thought of it in connection with this!

[John]

Thank you for your help guys.

Could I ask one further question?

I have converted the frequency matrix into a transition probability matrix.

I want to simulate a Markov chain. Normally I would do this in a statistics package in excel but I am interested in learning how one would do this in Matlab.

https://dl.dropbox.com/u/54057365/All/matlab.JPG

I created an image to illustrate the process.

Is there a way in Matlab to use the transition matrix and a random number to simulate for example 20 steps beginning with state 1?

I would appreciate any help.

Regards

John

[Roger Stafford]

Presumably you have normalized each row to sum to 1, getting a transition matrix, T. Just take this to the power of the number of steps, T^n, for n steps.

[John]

Hello Roger,

Yes, that is how I got the transition matrix.

Are you saying to simualte 20 steps in a markov chain that I take the transition matrix to the power of 20?

Would you have time to demonstrate this? I new to matlab and don't quite know how to do this, or if I follow you correctly?

Kind Regards

[Roger Stafford]

Yes, T^20. Let's suppose you have the transition matrix

 T = [ .3 , .7 ;
       .6 , .4 ]

Then T^2 = T*T would be

 T^2 = [ .3*.3+.7*.6 , .3*.7+.7*.4 ;
         .6*.3+.4*.6 , .6*.7+.4*.4 ]

     = [ .51 , .49 ;
         .42 , .58 ]

In other words, for example, given that you have started with 1 the probability that after two steps you will end with 2 (.49) is the probability that after one step you will remain at 1 (.3) times the conditional probability that on the next step you will go to 2 (.7) plus the probability that on the first step you will go to 2 (.7) times the conditional probability that on the next step you will remain at 2 (.4) giving a total of .3*.7+.7*.4 = .49 . This is just what matrix multiplication accomplishes.