1
$\begingroup$

Suppose we have an irreducible positive-recurrent Markov process $\{X(t), t\geq0\}$ with generator $G$. Let $P(t)$ be its transition probability matrix and $\pi$ its stationary distribution. Then we know that $\pi G=0$ and $P'(t)=P(t)G$ and hence $P'(t)\rightarrow 0$. I would like to have a more intuitive argument as to why $P'(t)$ should converge to 0. For example, suppose $\pi_j=\frac{1}{2}$, what is preventing us from having something like \begin{equation} p_{ij}(t)=\frac{1}{2}\frac{\sin{t^2}}{t}+\frac{1}{2}, \end{equation} so that $p_{ij}(t)\rightarrow \pi_j$, but $p'_{ij}(t)$ does not converge to 0.

$\endgroup$
2
  • $\begingroup$ I agree that your example function $\sin(t^2)/t$ converges to zero, but its derivative does not. However, the Markov chain relates $P(t)$ and $P'(t)$, so it avoids such things. Let's fix the initial condition and view $P(t)=(P_1(t), \ldots, P_N(t))$ as the vector of probabilities for each of the $N$ states at a given time $t$. We know $P'(t)^T = P(t)^TG$. So if $P(t)\rightarrow \pi$, then $P'(t)^T\rightarrow \pi^T G$. So $P'(t)$ converges to the fixed vector $\pi^TG$ and (since $P(t)$ converges, rather than growing linearly) it must be that $\pi^T G=0$. $\endgroup$
    – Michael
    May 29, 2015 at 17:47
  • $\begingroup$ Overall, if $P(t)$ converges to a fixed vector, then its derivative must converge to zero (due to the ODE structure $P'(t)^T = P(t)^TG$). So then I think your question reduces to "why does $P(t)$ converge to a fixed vector?" $\endgroup$
    – Michael
    May 29, 2015 at 17:50

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.