I'm looking for a reference which gives sufficient conditions for the variance to be positive in the central limit theorem for Markov chains (cf https://en.wikipedia.org/wiki/Markov_chain_central_limit_theorem). In other words, using the notation on the wikipedia page, I'm looking for a sufficient condition for $\sqrt{n}(\hat\mu_n-\mu)$ to legitimately be a random variable in the limit.

In my situation, the Markov chain is simply a random walk on a finite, aperiodic, strongly-connected graph, so a result relating to this simpler situation would be good enough if nothing more general exists. Also, in my situation, actually attempting to estimate $\sigma^2$ via the formula is pretty much out of the question.

I beleive that I can argue that the variance will be positive if there are two directed cycles $v_0\to v_2\to\dots\to v_n=v_0$ and $u_0\to u_2\to\dots\to u_m=u_0$ with ${1\over n}\sum_{t=1}^n g(v_t)\neq{1\over m}\sum_{t=1}^m g(u_t)$. Intuitively, this is because the chain will always have a chance to move around either of these cycles and these cycles contribute different values to the sum.

I'm not looking for anyone to write down a proof since, if necessary, I can likely prove it myself. However, I would really like to avoid having to spend the necessary pages to do so in a paper I'm writing. It seems very likely that such a result must have appeared in either a paper or book somewhere, but I haven't had any luck in hunting down a reference so far.