# Markov chain convergence problem.

Consider a markov chain matrix P of size n x n (n states).

P is known to be:

1- there are at least two absorbent states. one of them is denoted by null. (thus, we have that P_null,null = 1)

2- For the set of states that are not absorbent (called set H) , we have that P_h,null > 0 for all h in H.

3- Not all states are recurrent.

4- Aperiodic (the return to some states can occur at irregular times).

It is true that limit when n goes to infinity of P^n converges? Is this result well known or is the proof simple?

Thanks.

Yes. $P^n$ converges to a matrix $Q$ with

(i) $Q_{i,i}=1$ for each $i\not\in H$ ($i$ is absorbent), and

(ii) $\sum_{j\not\in H} Q_{i,j}=1$ for all $i\in H$.

To see (ii) we need that $\sum_{j\not\in H}P^n_{i,j}\rightarrow 1$ for all $i\in H$. For this note that $\sum_{j\not\in H}P^n_{i,j}$ is the probability of going from $i$ to an absorbent state in $n$ steps, and so if $P_{i,\text{null}}\ge\lambda>0$ for all $i\in H$ then for all $j\not\in H$, $$P^n_{i,j}\le (1-\lambda)^n\rightarrow 0.$$ To get a unique such $Q$ we need to show for each absorbent state (say, for null) $$\lim_n \ P^n_{i,\text{null}}\quad\text{exists}$$ for each $i\in H$. But $P^n_{i,\text{null}}\le P^{n+1}_{i,\text{null}}$ since once we get to null we stay there.

• ? Let's call another absorbent state foo. Can't we have in addition (1') $P_{i,\mathrm{foo}}>0$ for all $i \in H$ ?? Sep 9, 2010 at 18:15
• @Gerald Edgar: You're right. I fixed my answer in response to your comment. Sep 9, 2010 at 18:50
• The proof of uniqueness is well done? Not clear to me. Thanks. Sep 9, 2010 at 20:30

Yes, uniqueness holds.

Condition 2 implies that every state $j$ is either absorbing $(j\not\in H)$ or transient $(j\in H)$. Define the absorption time to be $T=\inf (n\geq 0: X_n\not\in H)$. This $T$ is almost surely finite for any starting state $i$, that is, the chain is eventually absorbed.

If $j\in H$, then $p^n_{ij}=P_i(X_n=j)\leq P_i(T>n)\to 0=:Q_{ij}$ as $n\to\infty$.

If $j\not\in H$, then $p^n_{ij}=P_i(X_n=j)\uparrow P_i(X_T=j)=:Q_{ij}$ as $n\to\infty$.