14
$\begingroup$

The following seems like such an elementary question, but I didn't get anywhere with it.

Suppose you are considering a Markov chain in continuous time which is transient and has an invariant measure (so one which explodes in finite time almost surely.) My question is: what does an invariant measure represent?

To fix ideas, consider the Markov chain on $S=\{0,1,\ldots\},$ which moves to the right with probability $2/3$ and to the left with probability $1/3$, and has total jump rate $q_i = 3^{i}$ at state $i \ge 1$. (To make computations nice, say that for $i=0$ the chain can only go to the right with total rate $q_0 = 2/3$).

Then it is elementary to check that this chain admits an invariant measure $$ \pi_i = \pi_0 (\frac{2}{3})^i, $$ obtained by solving the detailed balance equations. Of course the chain is also transient. Elementary Markov chain theory immediately implies that the chain is explosive, meaning that it will accumulate an infinite number of jumps in finite time almost surely.

The questions that is troubling me (even in such a basic example!) is the following: what does $\pi$ represent for this chain?

The following is a natural guess. Assume that the chain starts at $X_0$ distributed according to $\pi$. Then at the first explosion time $\zeta_0$, let the chain come back at an independent position also chosen according to $\pi$. Keep going in this fashion forever: each time the chain explodes, let it come back at a position sampled from $\pi$.

Question 1. Let $t>0$ be arbitrary. Is it true that $X_t$ has the law of $\pi$?

Question 2. If we start and restart the chain after each explosion with some given measure $\nu$ (which could be distinct from $\pi$), does $X_t$ converge in distribution to a certain measure $\mu$ as $t\to \infty$ (which could for instance be some mixture of $\pi$ and $\nu$) ?

Improve this question
$\endgroup$
4
  • $\begingroup$ Your invariant measure is wrong. To see this, note that state $j+1$ must be more probable than state $j-1$. $\endgroup$
    – Steve Huntsman
    Commented Aug 21, 2012 at 17:08
  • $\begingroup$ Since that is obviously impossible, there is no invariant measure. $\endgroup$
    – Will Sawin
    Commented Aug 21, 2012 at 17:19
  • 2
    $\begingroup$ Steve, Will, the MC is in continuous time ("has total jump rate $q_i=3^i$"), which means that it spends less time in the bigger integers, so in fact $\pi$ is the invariant probability measure. $\endgroup$
    – Ori Gurel-Gurevich
    Commented Aug 21, 2012 at 17:52
  • $\begingroup$ My bad, I had LaTeX rendering issues and thought the rate was 3 for each state. Sorry! $\endgroup$
    – Steve Huntsman
    Commented Aug 21, 2012 at 20:06

2 Answers 2

Reset to default
6
$\begingroup$

1) No, it won't. Suppose $\mu_j(t)$ is the probability vector at time $t$. Let explosions happen at rate $r(t)$ at time $t$. Then we should have $$ \dfrac{d}{dt} \mu_j(t) = -3^j \mu_j(t) + (2/3) 3^{j-1} \mu_{j-1}(t) + (1/3) 3^{j+1} \mu_{j+1}(t) + r(t) \pi_j $$ where the first term on the right represents jumps out of state $j$, the second and third represent jumps into this state from $j-1$ and $j+1$, and the fourth represents "resets" from explosions.
But if $\mu_j(t) = \pi_j$ for $t > 0$, the left side would be $0$ while the first three terms on the right would add to $0$ because of detailed balance, and the fourth term being positive would make the equation false. In order to have a stationary state in a system that includes an input to state $j$ from explosions, the terms involving jumps between $j-1, j, j+1$ would need to have a negative sum.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Dear Robert, Thanks for your comments and sorry for long time in response. I am not totally comfortable with the derivation of your equation. I always thought this process would satisfy the Kolmogorov backward and forward equations - without being the minimal solution. See, for instance, section 2.9 in James Norris' book on Markov chains. But you may well be right - in which case the question of "what is this invariant measure" is even more puzzling to me ! $\endgroup$
    – Nathanael Berestycki
    Commented Aug 25, 2012 at 7:46
0
$\begingroup$

It is incorrect to call the solution of the detailed balance equation that you found an invariant measure. No invariant measure exists for this process since it is transient as you have mentioned.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ It is a classical (and surprising) feature of continuous Markov chains that they can have an invariant measure while being transient. See, for instance, section 3.5 in James Norris' book on Markov chains. (Notice that the definition of invariant measure is the usual one and does not require the process to be recurrent or non-explosive). In any case, no matter how you would call such a measure, I hope you'll agree it is interesting to know what it means for the process... $\endgroup$
    – Nathanael Berestycki
    Commented Aug 25, 2012 at 7:49
  • 1
    $\begingroup$ @Nathanael Berestycki: A measure is invariant if it is preserved by the Markov semigroup (i.e., if you start the process with this initial distribution, then for any positive time the distribution stays the same). This measure is not preserved since the mass is driven to infinity and for any finite subset A of S, the probability that the process is in A goes to zero as time goes to infinity. $\endgroup$
    – Yuri Bakhtin
    Commented Aug 28, 2012 at 12:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .