# Convergence of stationary distributions of a sequence of Markov Chains

I fairly new in the field of Stochastic Processes and Markov Chains so excuse my ignorance.

My question is: If we have a sequence of Markov chains such that each one has a stationary distribution $$\pi^{(n)}$$ and the chains converge in some way to another Markov chain that has stationary distribution $$\pi$$, can we say that the $$\pi^{(n)}$$'s converge to $$\pi$$ (in some way)?

More precisely: Let $$G$$ be a simple (ie no loops or multiple edges), finite, connected graph. Suppose that we have a sequence of Markov chains over $$G$$. Let $$\boldsymbol{P}_1, \boldsymbol{P}_2, \dots$$ denote the corresponding transition matrices. Assume that all chains have a stationary distribution (for example, this can be guaranteed when the weights on each edge are positive since $$G$$ is connected), call them $$\pi^{(n)}$$. Now say that $$\boldsymbol{P}_n\to\boldsymbol{P}$$ in some way (for example, let's say that we have entry-wise almost sure convergence, or $$\|\boldsymbol{P}_n-\boldsymbol{P}\|\to 0$$ for some matrix norm). Suppose that $$\boldsymbol{P}$$ is a stochastic matrix with stationary distribution $$\pi$$. Then can we say that $$\pi^{(n)}\to\pi$$ in some way (similar to the way that the matrices converge)?

My feeling is that there should exist such theorems (maybe with some stronger assumptions). I tried to find such results but I was not successful. Can someone give a reference about such results?

We assume that the Markov chains are on a finite state space, that $$P_n \to P$$ pointwise, and the limit matrix $$P$$ is irreducible, so its stationary measure $$\pi$$ is unique. Let $$\pi^{(n_k)} \to \mu$$ be a convergent subsequence of $$\pi^{(n)}$$. Then $$\pi^{(n_k)}P_{n_k}=\pi^{(n_k)}$$, so continuity of multiplication implies that $$\mu P=\mu$$. Thus $$\mu=\pi$$. Since this holds for every convergent subsequence and the simplex of probability vectors is compact, we conclude that $$\pi^{(n)} \to \pi$$.
First $$\pi_nP_n^t = \pi_n$$ for all $$n,t$$. Since for the limiting matrix $$P$$ the distance to stationarity $$d(t) = \sup_\mu \|\mu P^t - \pi\|_{TV}$$ converges to 0 as $$t\to+\infty$$, there exists $$t_0$$ such that $$d(t_0)\le \epsilon$$. Then
\begin{align} \|\pi-\pi_n\|_{TV} &\le \|\pi - \pi_nP^{t_0}\|_{TV} + \|\pi_n P^{t_0} - \pi_n P_n^{t_0}\|_{TV} \\&\le \epsilon+ \sup_\mu \|\mu(P^{t_0}-P_n^{t_0})\|\|_{TV}. \end{align} Finally $$\sup_\mu \|\mu(P^{t_0}-P_n^{t_0})\|_{TV}\to 0$$ as $$n\to+\infty$$ for a fixed $$t_0$$ if you assume that $$P_n\to P$$ entrywise.
• Why does $\mu P^t$ converge to $\pi$ as $t \to \infty$? I think $P$ wasn't assumed to be aperiodic. (But probably one can replace the transition matrices $P^t$ with their Cesàro means to make the argument also work in the case where $P$ is not aperiodic). Oct 17 '21 at 7:43
• @Jochen Glueck Alternatively you can replace $P$ with the lazy chain $(P+I)/2$ and similarly for $P_n$. Oct 17 '21 at 19:33
• Nice proof! Unlike the (slightly shorter) proof via compactness, this proof has the advantage of potentially giving a rate of convergence by optimizing $t_0(\epsilon)$. Oct 18 '21 at 17:31