1
$\begingroup$

I have a Markov Chain of $N$ states. Such states represent the energy levels in a molecule.

The states' connectivity is as follows:

  1. States $j\in\{0,\ldots,N\}$ transition to $k\in\{\max(j-M,0),...,\min(j+M,N)\}\setminus j$, with $N\gg M$. The resulting transition matrix $\mathbf{P}$ has known terms $P_{jk}$, and it is a hollowed band matrix where detailed balance is followed.

  2. States $l\in\{L,...,N\}$ ALSO transition outside the Markov Chain, with rates $\mu_l>0$. Every state $l$ retains transitions to the $k$ states described in the previous point. For instance, state $L+1$ has both $j$ and $l$ transitions.

I am interested in recovering the stationary distribution $\tilde\pi_j$ of such Markov Chain.

If $\mu_l=0$, the Markov Chain has stationary distribution $\pi_j$ known from detailed balance and probability closure condition: $$ \pi_kP_{kj}=\pi_{j}P_{jk},\quad \sum_j\pi_j=1 $$ writing $\pi_j=\pi_0P_{0j}/P_{j0}$ and substituting into $\sum_j\pi_j=1$, one obtains $\pi_0$ and all the remaining values.

Now, when $\mu_l>0$, the stationary distribution $\tilde\pi_j$ is different. To tackle the problem, I have assumed that the $l$ states go to an absorbing state $J=N+1$, thus the transition matrix gains an absorbing state (one row and one column) with: $$ P_{lJ}=\mu_l,\quad P_{(j\setminus l)J}=0,\quad P_{Jl}=0,\quad P_{J(j\setminus l)}=0,\quad P_{JJ}=1 $$ But from now, to the best of my knowledge, I do not know how to proceed. I have the feeling that by introducing $J$, I am rendering the Markov Chain not ergodic anymore (as all states are unreachable from $J$). I have the feeling that $\tilde \pi_j=\delta(j-J)$, having $\delta(x)$ as the Dirac Delta Function.

I have read various sources (Snell, Montroll, Othmer) on Markov Chains and queuing theory, but all examples were not as specific as the one I have described above.

$\endgroup$
6
  • 2
    $\begingroup$ You are right: the unique stationary distribution is concentrated in the absorbing state. This is quite intuitive: random walk on states 1 through $N$ will eventually lead to one of the states $L$ through $N$, and then there is a positive probability of jumping to the absorbing state in one step. Thus, the Markov chain will eventually reach the absorbing state. $\endgroup$ Nov 1, 2019 at 19:51
  • 1
    $\begingroup$ It can be worth it in cases like this to write a bit of code to simulate a bunch of random walks. $\endgroup$
    – usul
    Nov 2, 2019 at 13:56
  • $\begingroup$ it may be of interest that excluding the absorbing states the chain still behaves regularly, the distribution given no absorbtion will converge, and this is a consequence of the perron-frobenius theorem $\endgroup$
    – mike
    Nov 4, 2019 at 8:09
  • $\begingroup$ @MateuszKwaśnicki Crystal clear explanation $\endgroup$
    – TheVal
    Nov 16, 2019 at 16:55
  • 2
    $\begingroup$ You might be interested in the quasi-stationary distribution of such a Markov chain, see the corresponding Wikipedia page for example. $\endgroup$ Nov 16, 2019 at 20:44

1 Answer 1

0
$\begingroup$

As the comments state, the resulting Markov Chain must have equilibrium distribution $$ \tilde\pi_j = \delta(j-J) $$ since any random walk through the Chain will eventually lead to states $j\ge L$, and then to $J$. This is justified by the Perron-Frobenius Theorem applied to the transition matrix $\mathbf{P}$.

Alternatively, the inevitable eventuality that a random process will hit $j$ may be given by transitive subshift of finite type applied to the finite oriented graph $G$ generated by $\mathbf{P}$, or the adjacency matrix $\mathbf{A}(\mathbf{P})$ such that: $$ a_{ij}=H(p_{ij}) $$ where $H$ is the Heaviside function $H(x>0)=1$. Since $G$ is strongly connected (there is a sequence of edges from any one vertex to any other vertex), then subshift of finite type on $G$ is transitive.

There may be other tools to describe the existence of an equilibrium distribution for such processes, such as the Krylov-Bogolyubov Theorem.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .