1
$\begingroup$

Consider a markov chain matrix P of size n x n (n states).

P is known to be:

1- there are at least two absorbent states. one of them is denoted by null. (thus, we have that P_null,null = 1)

2- For the set of states that are not absorbent (called set H) , we have that P_h,null > 0 for all h in H.

3- Not all states are recurrent.

4- Aperiodic (the return to some states can occur at irregular times).

It is true that limit when n goes to infinity of P^n converges? Is this result well known or is the proof simple?

Thanks.

$\endgroup$

2 Answers 2

1
$\begingroup$

Yes. $P^n$ converges to a matrix $Q$ with

(i) $Q_{i,i}=1$ for each $i\not\in H$ ($i$ is absorbent), and

(ii) $\sum_{j\not\in H} Q_{i,j}=1$ for all $i\in H$.

To see (ii) we need that $\sum_{j\not\in H}P^n_{i,j}\rightarrow 1$ for all $i\in H$. For this note that $\sum_{j\not\in H}P^n_{i,j}$ is the probability of going from $i$ to an absorbent state in $n$ steps, and so if $P_{i,\text{null}}\ge\lambda>0$ for all $i\in H$ then for all $j\not\in H$, $$ P^n_{i,j}\le (1-\lambda)^n\rightarrow 0. $$ To get a unique such $Q$ we need to show for each absorbent state (say, for null) $$ \lim_n \ P^n_{i,\text{null}}\quad\text{exists} $$ for each $i\in H$. But $P^n_{i,\text{null}}\le P^{n+1}_{i,\text{null}}$ since once we get to null we stay there.

$\endgroup$
3
  • 1
    $\begingroup$ ? Let's call another absorbent state foo. Can't we have in addition (1') $P_{i,\mathrm{foo}}>0$ for all $i \in H$ ?? $\endgroup$ Sep 9, 2010 at 18:15
  • $\begingroup$ @Gerald Edgar: You're right. I fixed my answer in response to your comment. $\endgroup$ Sep 9, 2010 at 18:50
  • $\begingroup$ The proof of uniqueness is well done? Not clear to me. Thanks. $\endgroup$
    – Gerardo
    Sep 9, 2010 at 20:30
0
$\begingroup$

Yes, uniqueness holds.

Condition 2 implies that every state $j$ is either absorbing $(j\not\in H)$ or transient $(j\in H)$. Define the absorption time to be $T=\inf (n\geq 0: X_n\not\in H)$. This $T$ is almost surely finite for any starting state $i$, that is, the chain is eventually absorbed.

If $j\in H$, then $p^n_{ij}=P_i(X_n=j)\leq P_i(T>n)\to 0=:Q_{ij}$ as $n\to\infty$.

If $j\not\in H$, then $p^n_{ij}=P_i(X_n=j)\uparrow P_i(X_T=j)=:Q_{ij}$ as $n\to\infty$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.