Suppose we have an irreducible positive-recurrent Markov process $\{X(t), t\geq0\}$ with generator $G$. Let $P(t)$ be its transition probability matrix and $\pi$ its stationary distribution. Then we know that $\pi G=0$ and $P'(t)=P(t)G$ and hence $P'(t)\rightarrow 0$. I would like to have a more intuitive argument as to why $P'(t)$ should converge to 0. For example, suppose $\pi_j=\frac{1}{2}$, what is preventing us from having something like \begin{equation} p_{ij}(t)=\frac{1}{2}\frac{\sin{t^2}}{t}+\frac{1}{2}, \end{equation} so that $p_{ij}(t)\rightarrow \pi_j$, but $p'_{ij}(t)$ does not converge to 0.
$\begingroup$
$\endgroup$
2
-
$\begingroup$ I agree that your example function $\sin(t^2)/t$ converges to zero, but its derivative does not. However, the Markov chain relates $P(t)$ and $P'(t)$, so it avoids such things. Let's fix the initial condition and view $P(t)=(P_1(t), \ldots, P_N(t))$ as the vector of probabilities for each of the $N$ states at a given time $t$. We know $P'(t)^T = P(t)^TG$. So if $P(t)\rightarrow \pi$, then $P'(t)^T\rightarrow \pi^T G$. So $P'(t)$ converges to the fixed vector $\pi^TG$ and (since $P(t)$ converges, rather than growing linearly) it must be that $\pi^T G=0$. $\endgroup$– MichaelMay 29, 2015 at 17:47
-
$\begingroup$ Overall, if $P(t)$ converges to a fixed vector, then its derivative must converge to zero (due to the ODE structure $P'(t)^T = P(t)^TG$). So then I think your question reduces to "why does $P(t)$ converge to a fixed vector?" $\endgroup$– MichaelMay 29, 2015 at 17:50
Add a comment
|