0
$\begingroup$

I fairly new in the field of Stochastic Processes and Markov Chains so excuse my ignorance.

My question is: If we have a sequence of Markov chains such that each one has a stationary distribution $\pi^{(n)}$ and the chains converge in some way to another Markov chain that has stationary distribution $\pi$, can we say that the $\pi^{(n)}$'s converge to $\pi$ (in some way)?

More precisely: Let $G$ be a simple (ie no loops or multiple edges), finite, connected graph. Suppose that we have a sequence of Markov chains over $G$. Let $\boldsymbol{P}_1, \boldsymbol{P}_2, \dots$ denote the corresponding transition matrices. Assume that all chains have a stationary distribution (for example, this can be guaranteed when the weights on each edge are positive since $G$ is connected), call them $\pi^{(n)}$. Now say that $\boldsymbol{P}_n\to\boldsymbol{P}$ in some way (for example, let's say that we have entry-wise almost sure convergence, or $\|\boldsymbol{P}_n-\boldsymbol{P}\|\to 0$ for some matrix norm). Suppose that $\boldsymbol{P}$ is a stochastic matrix with stationary distribution $\pi$. Then can we say that $\pi^{(n)}\to\pi$ in some way (similar to the way that the matrices converge)?

My feeling is that there should exist such theorems (maybe with some stronger assumptions). I tried to find such results but I was not successful. Can someone give a reference about such results?

$\endgroup$
1
5
$\begingroup$

We assume that the Markov chains are on a finite state space, that $P_n \to P$ pointwise, and the limit matrix $P$ is irreducible, so its stationary measure $\pi$ is unique. Let $\pi^{(n_k)} \to \mu$ be a convergent subsequence of $\pi^{(n)}$. Then $\pi^{(n_k)}P_{n_k}=\pi^{(n_k)}$, so continuity of multiplication implies that $\mu P=\mu$. Thus $\mu=\pi$. Since this holds for every convergent subsequence and the simplex of probability vectors is compact, we conclude that $\pi^{(n)} \to \pi$.

$\endgroup$
2
$\begingroup$

First $\pi_nP_n^t = \pi_n$ for all $n,t$. Since for the limiting matrix $P$ the distance to stationarity $d(t) = \sup_\mu \|\mu P^t - \pi\|_{TV}$ converges to 0 as $t\to+\infty$, there exists $t_0$ such that $d(t_0)\le \epsilon$. Then

\begin{align} \|\pi-\pi_n\|_{TV} &\le \|\pi - \pi_nP^{t_0}\|_{TV} + \|\pi_n P^{t_0} - \pi_n P_n^{t_0}\|_{TV} \\&\le \epsilon+ \sup_\mu \|\mu(P^{t_0}-P_n^{t_0})\|\|_{TV}. \end{align} Finally $\sup_\mu \|\mu(P^{t_0}-P_n^{t_0})\|_{TV}\to 0$ as $n\to+\infty$ for a fixed $t_0$ if you assume that $P_n\to P$ entrywise.

$\endgroup$
3
  • $\begingroup$ Why does $\mu P^t$ converge to $\pi$ as $t \to \infty$? I think $P$ wasn't assumed to be aperiodic. (But probably one can replace the transition matrices $P^t$ with their Cesàro means to make the argument also work in the case where $P$ is not aperiodic). $\endgroup$ Oct 17 '21 at 7:43
  • 1
    $\begingroup$ @Jochen Glueck Alternatively you can replace $P$ with the lazy chain $(P+I)/2$ and similarly for $P_n$. $\endgroup$ Oct 17 '21 at 19:33
  • 1
    $\begingroup$ Nice proof! Unlike the (slightly shorter) proof via compactness, this proof has the advantage of potentially giving a rate of convergence by optimizing $t_0(\epsilon)$. $\endgroup$ Oct 18 '21 at 17:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.