1
$\begingroup$

Suppose I have a family of countable state-space, discrete-time Markov chains, indexed by a parameter $r \in \mathbb{R}$. The state space is the same for all values of $r$; the transition probabilities depend on $r$ continuously. Is there some result like:

If the chain obtained for $r=r_0$ is positive recurrent, then the same is true for all values of $r$ in some open neighborhood of $r_0$; and as $r \to r_0$, the associated invariant distributions converge. (I assume that if the first statement is true, then so is the second one.)

In terms of simple linear algebra: I have a countable set of linear equations (defining the invariant distribution in terms of the transition probabilities of the Markov chain), which depend continuously on $r$. At some value $r=r_0$ the system has a unique positive solution. Is there a neighborhood of $r_0$ for which this will still be true? (What conditions do I need to impose on my set of equations to get this?)

Improve this question
$\endgroup$
1
  • $\begingroup$ Did you look at Doeblin condition? Don't have a proof, but I think it may help you. $\endgroup$
    – LazyCat
    Commented Jan 30, 2012 at 23:45

2 Answers 2

Reset to default
4
$\begingroup$

No it's not true. Consider Markov chains on $\mathbb N$ where the only transitions are from $n$ to $n\pm 1$.

For $n>1$, set $P_{n,n+1}=r+\frac12(1-\frac1n)$ and $P_{n,n-1}=-r+\frac12(1+\frac1n)$.

For $r=0$, solving the detailed balance equation, we get $\pi_{n+1}n(n+2)=\pi_n(n-1)(n+1)$ which has solutions $\pi_n=C/[(n-1)(n+1)]$. Hence this is positive recurrent.

For $r>0$, there is a rightwards drift and hence it is transient.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Very nice! But is there a version of the statement that is true? (For example, in my MC all transition rates belong to a finite set, which means that you can't reproduce this construction in it.) $\endgroup$
    – Elena Yudovina
    Commented Jan 29, 2012 at 8:41
  • 1
    $\begingroup$ I doubt that anything much could be said at this kind of level of generality. Even if the transition rates belong to a finite set, I would expect that you could cook up a Markov chain on the integers, where each state could go to 3 other states with probability $p_1(r)$, $p_2(r)$ and $p_3(r)$. The things you would have to play with in an example like this would be $f_1(k)$, $f_2(k)$ and $f_3(k)$ the places you go from state $k$. My guess would be by a clever choice of these guys you could make it barely positive recurrent for $r=0$, but transient for $r>0$. $\endgroup$
    – Anthony Quas
    Commented Jan 29, 2012 at 8:50
0
$\begingroup$

Something like this is dealt with in the paper "Perturbation Theory and Finite Markov chains" by Schweitzer. (https://www.jstor.org/stable/3212261). In particular, see section 6.

The results are limited to finite markov chains that have a single irreducible subset of states (a single closed communicating class). The author shows that if all the transition probabilities vary continuously with respect to some parameter, the resulting invariant distribution will also vary continuously.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you, I'll look at that. However, the statement "eigenvalues of a finite matrix are continuous in the matrix coefficients" is much less surprising (and some versions of it are obvious), unlike anything similar for a countable state space. $\endgroup$
    – Elena Yudovina
    Commented Feb 6, 2018 at 15:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .