Markov chains: invariant measures and explosion - MathOverflow

Markov chains: invariant measures and explosion

2012-08-21T16:47:18Z

The following seems like such an elementary question, but I didn't get anywhere with it.

Suppose you are considering a Markov chain in continuous time which is transient and has an invariant measure (so one which explodes in finite time almost surely.) My question is: what does an invariant measure represent?

To fix ideas, consider the Markov chain on $S={0,1,\ldots},$ which moves to the right with probability $2/3$ and to the left with probability $1/3$, and has total jump rate $q_i = 3^{i}$ at state $i \ge 1$. (To make computations nice, say that for $i=0$ the chain can only go to the right with total rate $q_0 = 2/3$).

Then it is elementary to check that this chain admits an invariant measure $$ \pi_i = \pi_0 (\frac{2}{3})^i, $$ obtained by solving the detailed balance equations. Of course the chain is also transient. Elementary Markov chain theory immediately implies that the chain is explosive, meaning that it will accumulate an infinite number of jumps in finite time almost surely.

The questions that is troubling me (even in such a basic example!) is the following: what does $\pi$ represent for this chain?

The following is a natural guess. Assume that the chain starts at $X_0$ distributed according to $\pi$. Then at the first explosion time $\zeta_0$, let the chain come back at an independent position also chosen according to $\pi$. Keep going in this fashion forever: each time the chain explodes, let it come back at a position sampled from $\pi$.

Question 1. Let $t>0$ be arbitrary. Is it true that $X_t$ has the law of $\pi$?

Question 2. If we start and restart the chain after each explosion with some given measure $\nu$ (which could be distinct from $\pi$), does $X_t$ converge in distribution to a certain measure $\mu$ as $t\to \infty$ (which could for instance be some mixture of $\pi$ and $\nu$) ?

Answer by Robert Israel for Markov chains: invariant measures and explosion

2012-08-21T18:56:09Z

1) No, it won't. Suppose $\mu_j(t)$ is the probability vector at time $t$. Let explosions happen at rate $r(t)$ at time $t$. Then we should have $$ \dfrac{d}{dt} \mu_j(t) = -3^j \mu_j(t) + (2/3) 3^{j-1} \mu_{j-1}(t) + (1/3) 3^{j+1} \mu_{j+1}(t) + r(t) \pi_j $$ where the first term on the right represents jumps out of state $j$, the second and third represent jumps into this state from $j-1$ and $j+1$, and the fourth represents "resets" from explosions.
But if $\mu_j(t) = \pi_j$ for $t > 0$, the left side would be $0$ while the first three terms on the right would add to $0$ because of detailed balance, and the fourth term being positive would make the equation false. In order to have a stationary state in a system that includes an input to state $j$ from explosions, the terms involving jumps between $j-1, j, j+1$ would need to have a negative sum.

Answer by Yuri Bakhtin for Markov chains: invariant measures and explosion

2012-08-23T18:44:58Z

It is incorrect to call the solution of the detailed balance equation that you found an invariant measure. No invariant measure exists for this process since it is transient as you have mentioned.