Welcome to MathOverflow

Question

1

Suppose I have a family of countable state-space, discrete-time Markov chains, indexed by a parameter $r \in \mathbb{R}$. The state space is the same for all values of $r$; the transition probabilities depend on $r$ continuously. Is there some result like:

If the chain obtained for $r=r_0$ is positive recurrent, then the same is true for all values of $r$ in some open neighborhood of $r_0$; and as $r \to r_0$, the associated invariant distributions converge. (I assume that if the first statement is true, then so is the second one.)

In terms of simple linear algebra: I have a countable set of linear equations (defining the invariant distribution in terms of the transition probabilities of the Markov chain), which depend continuously on $r$. At some value $r=r_0$ the system has a unique positive solution. Is there a neighborhood of $r_0$ for which this will still be true? (What conditions do I need to impose on my set of equations to get this?)

flag	asked Jan 27 2012 at 11:30 Elena Yudovina 241●1●4

Anthony Quas · Accepted Answer · 2012-01-27 18:18:32Z UTC

4

No it's not true. Consider Markov chains on $\mathbb N$ where the only transitions are from $n$ to $n\pm 1$.

For $n>1$, set $P_{n,n+1}=r+\frac12(1-\frac1n)$ and $P_{n,n-1}=-r+\frac12(1+\frac1n)$.

For $r=0$, solving the detailed balance equation, we get $\pi_{n+1}n(n+2)=\pi_n(n-1)(n+1)$ which has solutions $\pi_n=C/[(n-1)(n+1)]$. Hence this is positive recurrent.

For $r>0$, there is a rightwards drift and hence it is transient.

edited Jan 27 2012 at 18:18

answered Jan 27 2012 at 17:55

Anthony Quas
6,315●1●12●29

Very nice! But is there a version of the statement that is true? (For example, in my MC all transition rates belong to a finite set, which means that you can't reproduce this construction in it.) – Elena Yudovina Jan 29 2012 at 8:41

1

I doubt that anything much could be said at this kind of level of generality. Even if the transition rates belong to a finite set, I would expect that you could cook up a Markov chain on the integers, where each state could go to 3 other states with probability $p_1(r)$, $p_2(r)$ and $p_3(r)$. The things you would have to play with in an example like this would be $f_1(k)$, $f_2(k)$ and $f_3(k)$ the places you go from state $k$. My guess would be by a clever choice of these guys you could make it barely positive recurrent for $r=0$, but transient for $r>0$. – Anthony Quas Jan 29 2012 at 8:50

LazyCat · Answer 2 · 2012-01-30 23:45:12Z UTC

-1

Did you look at Doeblin condition? Don't have a proof, but I think it may help you.

answered Jan 30 2012 at 23:45

LazyCat
86●1

Continuous family of Markov chains

Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)

2 Answers

You can accept an answer to one of your own questions by clicking the check mark next to it. This awards 15 reputation points to the person who answered and 2 reputation points to you.