I'd appreciate some help on a Markov chain result I'm trying to show. I believe the following is sufficient for a continuous time Markov chain $(X_t)$ with a countable state space to be ergodic:

  1. $(X_t)$ is irreducible.
  2. There exists a finite subset $A$ of states such that, for all $a\in A$, conditional on $X_0 = a$, the expected return time back to $A$ is integrable, that is, $E_a[R_A]<\infty$ where $$ R_A := \inf ( t > 0 \mid X_t \in A ). $$
  3. There exist $k,p>0$ such that for all $a,b \in A$, we have $$P (X_{t+k} = b \mid X_t = a) \geq p.$$

In other words, I'm under the impression that (2) and (3) implies positive recurrence. This is because if I start at any $a\in A$, then I take some finite time to return back to $A$. Then use the $k$ (finite no. of) steps to return back to $a$ to show postive recurrence. Whence (1) and (2)+(3) imply ergodicity.

Can some help show this claim or suggest why it's wrong?

Thanks Apus

  • $\begingroup$ I am not sure, but isn't it that ergodicity of ctMC is equivalent to the ergodicity of the underlying discrete-time Markov Chain? $\endgroup$
    – Ilya
    Jan 27, 2012 at 12:12
  • $\begingroup$ I guess that in 2. you mean $\inf\{t>0:X_t\in A\}$. But if $X_0\in A$, this infimum is almost surely $0$ (hence finite), for every continuous time Markov chain, since $X_0\in A$ and $\inf\{t>0:X_t\ne X_0\}$ is almost surely positive. $\endgroup$
    – Did
    Jan 28, 2012 at 15:05
  • $\begingroup$ @Ilya I don't think that's the case. Perhaps I can find a proof somewhere. @Didier Thanks for spotting the typo. Could you explain what you mean a little more please? $\endgroup$
    – A Chuh
    Jan 29, 2012 at 10:42
  • $\begingroup$ What part of my previous comment is not clear to you? (Your revised 2. is not correct, I will modify it.) $\endgroup$
    – Did
    Jan 29, 2012 at 19:39

1 Answer 1


Theorem 3.5.3: Let the intensity matrix $Q$ be irreducible. Then if some state is positive recurrent, every state is recurrent. Further there exists a stationary distribution $\lambda$.

-Some states (all in $A$) are positive recurrent (i.e. return time has finite expectation) which implies that all states are positive recurrent and that there exists a stationary distribution.

Please define what you mean by ergodic. For example there is a theorem which states that for a irreducible positive recurrent chain,

Theorem 3.8.1: $$\mathbb{P}\Big(\frac{1}{t} \int_0^t f(X_s)\,ds \rightarrow \bar{f} \text{ as } t \rightarrow\infty\Big)=1,$$ where $\bar{f} = \sum \lambda_j f_j$, and $\lambda$ is the stationary distribution.

But sometimes ergodic means that the limit of the probability exists.

Source: "Markov Chains" by Norris


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.