Skip to main content
Post Undeleted by Bjørn Kjos-Hanssen
added 49 characters in body
Source Link
Bjørn Kjos-Hanssen
  • 24.8k
  • 3
  • 58
  • 114

The hypotheses you have now don't seem to rule out that here are 2Suppose the answer is Yes. Then suppose we add two more states $i\ne j$ with $P_{i,j}=1$ and $P_{j,i}=1$. In this case $P^n$ does not converge because, and no other state can go to states $P^n_{i,i}$ is 0$i$ or 1 depending on whether $n$$j$. Then for the new matrix the assumptions are still satisfied, but now the answer is odd or evenNo. Therefore the answer must be No.

The hypotheses you have now don't seem to rule out that here are 2 states $i\ne j$ with $P_{i,j}=1$ and $P_{j,i}=1$. In this case $P^n$ does not converge because $P^n_{i,i}$ is 0 or 1 depending on whether $n$ is odd or even.

Suppose the answer is Yes. Then suppose we add two more states $i\ne j$ with $P_{i,j}=1$ and $P_{j,i}=1$, and no other state can go to states $i$ or $j$. Then for the new matrix the assumptions are still satisfied, but now the answer is No. Therefore the answer must be No.

Post Deleted by Bjørn Kjos-Hanssen
deleted 296 characters in body
Source Link
Bjørn Kjos-Hanssen
  • 24.8k
  • 3
  • 58
  • 114

Yes, $P^n$ convergesThe hypotheses you have now don't seem to the matrix Q withrule out that here are 2 states $Q_{i,j}=0$ if$i\ne j$ with $j\ne\text{null}$$P_{i,j}=1$ and $Q_{i,j}=1$ if $j=\text{null}$$P_{j,i}=1$. This isIn this case $P^n$ does not converge because $P^n_{i,j}$$P^n_{i,i}$ is the probability that if started in $i$ after0 or 1 depending on whether $n$ moves we are in state $j$. Take $\lambda>0$ such that $P_{i,\text{null}}\ge\lambda$ for all $i$. Then $P^n_{i,j}\le (1-\lambda)^n\longrightarrow 0$ for all $j\ne\text{null}$.

(Note that the question is not about almost sure behavior but about a limit of matrices, so Borel-Cantelli is not relevant to the question as statedodd or even.)

Yes, $P^n$ converges to the matrix Q with $Q_{i,j}=0$ if $j\ne\text{null}$ and $Q_{i,j}=1$ if $j=\text{null}$. This is because $P^n_{i,j}$ is the probability that if started in $i$ after $n$ moves we are in state $j$. Take $\lambda>0$ such that $P_{i,\text{null}}\ge\lambda$ for all $i$. Then $P^n_{i,j}\le (1-\lambda)^n\longrightarrow 0$ for all $j\ne\text{null}$.

(Note that the question is not about almost sure behavior but about a limit of matrices, so Borel-Cantelli is not relevant to the question as stated.)

The hypotheses you have now don't seem to rule out that here are 2 states $i\ne j$ with $P_{i,j}=1$ and $P_{j,i}=1$. In this case $P^n$ does not converge because $P^n_{i,i}$ is 0 or 1 depending on whether $n$ is odd or even.

Source Link
Bjørn Kjos-Hanssen
  • 24.8k
  • 3
  • 58
  • 114

Yes, $P^n$ converges to the matrix Q with $Q_{i,j}=0$ if $j\ne\text{null}$ and $Q_{i,j}=1$ if $j=\text{null}$. This is because $P^n_{i,j}$ is the probability that if started in $i$ after $n$ moves we are in state $j$. Take $\lambda>0$ such that $P_{i,\text{null}}\ge\lambda$ for all $i$. Then $P^n_{i,j}\le (1-\lambda)^n\longrightarrow 0$ for all $j\ne\text{null}$.

(Note that the question is not about almost sure behavior but about a limit of matrices, so Borel-Cantelli is not relevant to the question as stated.)