Suppose the answer is Yes. Then suppose we add two more states $i\ne j$ with $P_{i,j}=1$ and $P_{j,i}=1$, and no other state can go to states $i$ or $j$. Then for the new matrix the assumptions are still satisfied, but now the answer is No. Therefore the answer must be No.
Bjørn Kjos-Hanssen
- 24.8k
- 3
- 58
- 114