Quantitative version of ergodic theorem in Markov chains

Question

Consider an irreducible Markov chain $X_t$ with finite state space $E$, and unique invariant measure $\pi$. Fix a function $V:E\to\mathbb R$ such that $E_\pi[V]=0$. The ergodic theorem tells us that, almost surely, $$\frac{1}{t}\int_0^t V(X_s)ds \to E_\pi[V]=0. $$

My question is, is there an estimate on the rate of convergence in $L^p$(at least for $p=1,2$)? Namely, can we bound the $L^p$ norm of $$\frac{1}{t}\int_0^t V(X_s)ds$$ in terms of $C(V)t^{-m}$? where $m$ is some constant related to the mixing property of the chain $X_t$, like mixing time or spectral gap.

Answer 1

At least in $L^2$ this question has been studied in the context of Markov central limit theorems. Let $\mathbb P_\pi$ be the induced measure on path space when starting according to $\pi$. We have $$\mathrm{Var}_{\mathbb P_\pi}\left( \frac 1t \int_0^t V(X_s)\,ds \right) = \frac 2t \int_0^t \left(1 - \frac rt\right)\mathrm{Cov}_{\mathbb P_\pi}[V(X_0),V(X_r)]\,dr$$ Now one needs to obtain fast enough decaying bounds for the covariances, to obtain something like $\le C/t$ as suggested in your question. Since your state space is finite you will hopefully be able to obtain quite strong mixing results.

Geometrically fast mixing is one easy strong condition. If you have a reversible Markov process there are also connections to the spectral gap. Section 3.4 of these lecture notes by Eberle provides an overview. A quick search for references on functional central limit theorems (which requires at least asymptotical results on this quantity) makes me suggest the following:

Liu, Y., Zhang, Y. Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes. Front. Math. China 10, 933–947 (2015). https://doi.org/10.1007/s11464-015-0488-5