The hypotheses you have now don't seem to rule out that here are 2Suppose the answer is Yes. Then suppose we add two more states $i\ne j$ with $P_{i,j}=1$ and $P_{j,i}=1$. In this case $P^n$ does not converge because, and no other state can go to states $P^n_{i,i}$ is 0$i$ or 1 depending on whether $n$$j$. Then for the new matrix the assumptions are still satisfied, but now the answer is odd or evenNo. Therefore the answer must be No.