Two state markov chain realization

I have a state transition probability matrix and a state probability vector

[0.9 0.1; 0.1 0.9] & [0.4 0.6] respectively.

Now, I want to generate the states according to this. say 100 state sequence.

Any sort of help would be appreciated.

Thanks.

Answer 1

[the cyclist]

I have to admit that my memory of MC is a bit rusty. Does this do what you want? Do you expect a convergence to a state probability vector [0.5 0.5]?

If this is right, there are faster implementations, but I wanted to lay out the basics clearly.

transitionMatrix = [0.9 0.1; 0.1 0.9];

initialProbabilityState = [0.4; 0.6]; % Made it a column vector, rather than a row.

nStates = 100;

states = zeros(2,nStates);

states(:,1) = initialProbabilityState;

for ns = 2:nStates
    states(:,ns) = transitionMatrix*states(:,ns-1);
end

[the cyclist]

Given that this is homework, it would be best if you were to post what you yourself have tried to do, and where you are stuck. Then, maybe you can get some hints from someone about how to proceed. You'll learn more that way than if someone just does your assignment for you.

Answer 2

[Fangjun Jiang]

You've asked the same question multiple times. Sounds like you need to go back to your textbook to learn what is state transition probability and Markov chain.

[lemontree45]

Well, this is what I have done till now. I could generate the state transition probability matrix and state probability vector with the help of the set of sequence given.

Now I need to do the reverse of it. With the help of the transition probability matrix, I need to get the sequence back. As per the cyclist suggestion I tried that way, which doesn't give the same sequence.

For my defense, I have been trying. I get pretty confused with the probability stuffs. sorry for the silly questions.

Here is what I have coded till now.

s=load('stateseq'); % loading the file

s=s.stateseq; % s consists of states '1' or '2'

%% state probability vector

prob_1=sum(s==1)./length(s); % P(1)

prob_2=1-prob_1; % P(2)

state_prob_vector=[prob_1 prob_2]';

%% transition matrix

count=length(s); % total number of states in the sequence

s1=sum(s==1); % # of state '1'

s2=count-s1; % # of state '2'

p=zeros(2,2);

for i=1:length(s)-1

if s(i)==1 && s(i+1)==1

p(1,1)=p(1,1)+1;

else if s(i)==1 && s(i+1)==2

p(1,2)=p(1,2)+1;

else if s(i)==2 && s(i+1)==1

p(2,1)=p(2,1)+1;

else if s(i)==2 && s(i+1)==2

p(2,2)=p(2,2)+1;

end

end

end

end

end

p(1,1)=p(1,1)/s1;

p(1,2)=p(1,2)/s1;

p(2,1)=p(2,1)/s2;

p(2,2)=p(2,2)/s2;

%% state sequence generator

seq_gen=zeros(1,100);

state_prob_vector(:,1)=state_prob_vector;

if state_prob_vector(:,1)>0.5

seq_gen(1,1)=1;

else

seq_gen(1,1)=2;

end

for j=2:100

state_prob_vector(:,j)=p*state_prob_vector(:,j-1);

if state_prob_vector(:,j)>0.5

seq_gen(1,j)=1;

else

seq_gen(1,j)=2;

end

end

[Fangjun Jiang]

Your code looks good. And I apologize for the mistake that I made in my answer to your previous question. p=p/100 is incorrect. Should do like you did. p(1,1)=p(1,1)/s1 stuff.

Now let's debug your code. First I think you need to set an initial state. Because, s1+s2=100, there are 100 transitions. In your current code, there are 99 transitions. In your current code, before doing p(1,1)=p(1,1)/s1 stuff, you can check that sum(sum(p)) equals 99, which will later cause you probability matrix problem. Each row of the probability matrix should sum equals 1. Make sure you check that before doing the reversing.

[Fangjun Jiang]

You don't have to do the same as mine. Your code can achieve the same. But the important thing at the end of calculation p (or P in my case) is to make sure

>> sum(P,2)

ans =

1

1

Here is my code.

N=100;

Data=rand(N,1);

TF=Data>0.5;

Data(TF)=2;

Data(~TF)=1;

P=zeros(2,2);

FromState=1;

for k=1:N

ToState=Data(k);

P(FromState,ToState)=P(FromState,ToState)+1;

FromState=ToState;

end

P(2,:)=P(2,:)/sum(TF);

P(1,:)=P(1,:)/sum(~TF);

[lemontree45]

Thank you. yes. Initial state can be obtained from the state probability vector. I have taken care of that part. Hope that side is correct. Also, I am getting >>sum(P,2)=[1 1]. so I guess that part is clear too.

I couldn't find any problem with my state transition probability matrix and the state probability vector. But still I am not able to regenerate the same sequence when I do the reverse step.

Is it really possible to regenerate it correctly or would there be some sort of uncertainty?

[Fangjun Jiang]

I don't think you can re-generate the exact sequence. Think about your probability matrix, it is derived from 100 state transition. There are so many other slightly varied sequence that can draw to the same probability matrix.

When you generate the state probability vector, remember that you vector should be defined as row vector as V=[0.9 0.1], and the next vector would be V*P. This is different from cyclist because the definition of P(1,1), P(1,2),P(2,1),P(2,2).