Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Consider a markov chain matrix P of size n x n (n states).

P is known to be:

1- there are at least two absorbent states. one of them is denoted by null. (thus, we have that P_null,null = 1)

2- For the set of states that are not absorbent (called set H) , we have that P_h,null > 0 for all h in H.

3- Not all states are recurrent.

4- Aperiodic (the return to some states can occur at irregular times).

It is true that limit when n goes to infinity of P^n converges? Is this result well known or is the proof simple?

Thanks.

share|improve this question
add comment

2 Answers

Yes. $P^n$ converges to a matrix $Q$ with

(i) $Q_{i,i}=1$ for each $i\not\in H$ ($i$ is absorbent), and

(ii) $\sum_{j\not\in H} Q_{i,j}=1$ for all $i\in H$.

To see (ii) we need that $\sum_{j\not\in H}P^n_{i,j}\rightarrow 1$ for all $i\in H$. For this note that $\sum_{j\not\in H}P^n_{i,j}$ is the probability of going from $i$ to an absorbent state in $n$ steps, and so if $P_{i,\text{null}}\ge\lambda>0$ for all $i\in H$ then for all $j\not\in H$, $$ P^n_{i,j}\le (1-\lambda)^n\rightarrow 0. $$ To get a unique such $Q$ we need to show for each absorbent state (say, for null) $$ \lim_n \ P^n_{i,\text{null}}\quad\text{exists} $$ for each $i\in H$. But $P^n_{i,\text{null}}\le P^{n+1}_{i,\text{null}}$ since once we get to null we stay there.

share|improve this answer
1  
? Let's call another absorbent state foo. Can't we have in addition (1') $P_{i,\mathrm{foo}}>0$ for all $i \in H$ ?? –  Gerald Edgar Sep 9 '10 at 18:15
    
@Gerald Edgar: You're right. I fixed my answer in response to your comment. –  Bjørn Kjos-Hanssen Sep 9 '10 at 18:50
    
The proof of uniqueness is well done? Not clear to me. Thanks. –  Gerardo Sep 9 '10 at 20:30
add comment

Yes, uniqueness holds.

Condition 2 implies that every state $j$ is either absorbing $(j\not\in H)$ or transient $(j\in H)$. Define the absorption time to be $T=\inf (n\geq 0: X_n\not\in H)$. This $T$ is almost surely finite for any starting state $i$, that is, the chain is eventually absorbed.

If $j\in H$, then $p^n_{ij}=P_i(X_n=j)\leq P_i(T>n)\to 0=:Q_{ij}$ as $n\to\infty$.

If $j\not\in H$, then $p^n_{ij}=P_i(X_n=j)\uparrow P_i(X_T=j)=:Q_{ij}$ as $n\to\infty$.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.