Question

What is the difference between strong markov property and markov property?

Answer 1

Answer:

A stochastic process has the Markov property if the probabilistic behaviour of the chain in the future depends only on its present value and discards its past behaviour.

The strong Markov property is based on the same concept except that the time, say TT, that the present refers to is a random quantity with some special properties.

TT is called stopping time and it is a random variable taking values in {0,1,2,…}{0,1,2,…} such that any value T=nT=n can be determined completely by the values of the chain, X0,X1,…,XnX0,X1,…,Xn, up to time nn.

A very simple example is when you throw a coin and you want to stop when you reach T=nT=n heads. T=nT=n is completely determined by the values of the sequence of the previous tosses. Of course, TT is random.

The strong Markov property goes as follows. If TT is a stopping time, for m≥1m≥1

P(XT+m=j∣Xk=xk,0≤k<T;XT=i)=P(XT+m=j∣XT=i)

P(XT+m=j∣Xk=xk,0≤k<T;XT=i)=P(XT+m=j∣XT=i)

So conditionally on XT=iXT=i the chain again discards whatever happened previously to time TT.

In order to determine the(unconditional) probabilistic behaviour of a(homogeneous) Markov chain at time nn one needs to know the one step transition matrix and the marginal behaviour of XX at a previous time point, call it t=0t=0 without loss of generality. ie one should know P(X1=j∣X0=i)P(X1=j∣X0=i) and P(X0)