MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose I have a family of countable state-space, discrete-time Markov chains, indexed by a parameter $r \in \mathbb{R}$. The state space is the same for all values of $r$; the transition probabilities depend on $r$ continuously. Is there some result like:

If the chain obtained for $r=r_0$ is positive recurrent, then the same is true for all values of $r$ in some open neighborhood of $r_0$; and as $r \to r_0$, the associated invariant distributions converge. (I assume that if the first statement is true, then so is the second one.)

In terms of simple linear algebra: I have a countable set of linear equations (defining the invariant distribution in terms of the transition probabilities of the Markov chain), which depend continuously on $r$. At some value $r=r_0$ the system has a unique positive solution. Is there a neighborhood of $r_0$ for which this will still be true? (What conditions do I need to impose on my set of equations to get this?)

share|cite|improve this question
Did you look at Doeblin condition? Don't have a proof, but I think it may help you. – LazyCat Jan 30 '12 at 23:45
up vote 4 down vote accepted

No it's not true. Consider Markov chains on $\mathbb N$ where the only transitions are from $n$ to $n\pm 1$.

For $n>1$, set $P_{n,n+1}=r+\frac12(1-\frac1n)$ and $P_{n,n-1}=-r+\frac12(1+\frac1n)$.

For $r=0$, solving the detailed balance equation, we get $\pi_{n+1}n(n+2)=\pi_n(n-1)(n+1)$ which has solutions $\pi_n=C/[(n-1)(n+1)]$. Hence this is positive recurrent.

For $r>0$, there is a rightwards drift and hence it is transient.

share|cite|improve this answer
Very nice! But is there a version of the statement that is true? (For example, in my MC all transition rates belong to a finite set, which means that you can't reproduce this construction in it.) – Elena Yudovina Jan 29 '12 at 8:41
I doubt that anything much could be said at this kind of level of generality. Even if the transition rates belong to a finite set, I would expect that you could cook up a Markov chain on the integers, where each state could go to 3 other states with probability $p_1(r)$, $p_2(r)$ and $p_3(r)$. The things you would have to play with in an example like this would be $f_1(k)$, $f_2(k)$ and $f_3(k)$ the places you go from state $k$. My guess would be by a clever choice of these guys you could make it barely positive recurrent for $r=0$, but transient for $r>0$. – Anthony Quas Jan 29 '12 at 8:50

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.