Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Hi,

I'd appreciate some help on a Markov chain result I'm trying to show. I believe the following is sufficient for a continuous time Markov chain $(X_t)$ with a countable state space to be ergodic:

  1. $(X_t)$ is irreducible.
  2. There exists a finite subset $A$ of states such that, for all $a\in A$, conditional on $X_0 = a$, the expected return time back to $A$ is integrable, that is, $E_a[R_A]<\infty$ where $$ R_A := \inf ( t > 0 \mid X_t \in A ). $$
  3. There exist $k,p>0$ such that for all $a,b \in A$, we have $$P (X_{t+k} = b \mid X_t = a) \geq p.$$

In other words, I'm under the impression that (2) and (3) implies positive recurrence. This is because if I start at any $a\in A$, then I take some finite time to return back to $A$. Then use the $k$ (finite no. of) steps to return back to $a$ to show postive recurrence. Whence (1) and (2)+(3) imply ergodicity.

Can some help show this claim or suggest why it's wrong?

Thanks Apus

share|improve this question
    
I am not sure, but isn't it that ergodicity of ctMC is equivalent to the ergodicity of the underlying discrete-time Markov Chain? –  Ilya Jan 27 '12 at 12:12
    
I guess that in 2. you mean $\inf\{t>0:X_t\in A\}$. But if $X_0\in A$, this infimum is almost surely $0$ (hence finite), for every continuous time Markov chain, since $X_0\in A$ and $\inf\{t>0:X_t\ne X_0\}$ is almost surely positive. –  Did Jan 28 '12 at 15:05
    
@Ilya I don't think that's the case. Perhaps I can find a proof somewhere. @Didier Thanks for spotting the typo. Could you explain what you mean a little more please? –  A Chuh Jan 29 '12 at 10:42
    
What part of my previous comment is not clear to you? (Your revised 2. is not correct, I will modify it.) –  Did Jan 29 '12 at 19:39
add comment

1 Answer

Theorem 3.5.3: Let the intensity matrix $Q$ be irreducible. Then if some state is positive recurrent, every state is recurrent. Further there exists a stationary distribution $\lambda$.

-Some states (all in $A$) are positive recurrent (i.e. return time has finite expectation) which implies that all states are positive recurrent and that there exists a stationary distribution.

Please define what you mean by ergodic. For example there is a theorem which states that for a irreducible positive recurrent chain,

Theorem 3.8.1: $$P \bigg(\frac{1}{t} \int_0^t f(X_s)\,ds \rightarrow \bar{f} \text{as} t \rightarrow\infty\bigg)=1,$$ where $\bar{f} = \sum \lambda_j f_j$, and $\lambda$ is the stationary distribution.

But sometimes ergodic means that the limit of the probability exists.

Source: "Markov Chains" by Norris

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.