Question

Define probability transition matrix signal coding and estimation theory

Answer 1

We often list the transition probabilities in a matrix. The matrix is called the state transition matrixor transition probability matrix and is usually shown by PP. Assuming the states are 11, 22, ⋯⋯, rr, then the state transition matrix is given by

P=⎡⎣⎢⎢⎢⎢⎢⎢⎢⎢p11p21...pr1p12p22...pr2............p1rp2r...prr⎤⎦⎥⎥⎥⎥⎥⎥⎥⎥.P=[p11p12...p1rp21p22...p2r............pr1pr2...prr].

Note that pij≥0pij≥0, and for all ii, we have

∑k=1rpik=∑k=1rP(Xm+1=k|Xm=i)=1.∑k=1rpik=∑k=1rP(Xm+1=k|Xm=i)=1.

This is because, given that we are in state ii, the next state must be one of the possible states. Thus, when we sum over all the possible values of kk, we should get one. That is, the rows of any state transition matrix must sum to one.

State Transition Diagram:

A Markov chain is usually shown by a state transition diagram. Consider a Markov chain with three possible states 11, 22, and 33 and the following transition probabilities

P=⎡⎣⎢⎢⎢⎢1413121200142312⎤⎦⎥⎥⎥⎥.P=[1412141302312012].

Figure 11.7 shows the state transition diagram for the above Markov chain. In this diagram, there are three possible states 11, 22, and 33, and the arrows from each state to other states show the transition probabilities pijpij. When there is no arrow from state ii to state jj, it means that pij=0pij=0.Figure 11.7 - A state transition diagram.