We assume the Markov chain to be countable state space, timehomogeneous. Does it necessarily have a stationary distribution? I found a paper on arXiv.org (http://arxiv.org/abs/math/0610707) that proves that for every continuous transformation from the standard infinite dimensional (the convex hull of the standard bases of $\mathbb{R}^\infty$) simplex to itself has a fixed point. So I guess it necessarily has but when I look for such a theorem on books, I cannot find one. Thank you for your help!
1

2$\begingroup$ How about the Markov chain on $\mathbb Z$ defined by $\mathbb P(X_{n+1}=k+1X_n=k)=1$? $\endgroup$ – Anthony Quas Apr 20 '13 at 9:22

$\begingroup$ The arxiv paper you cite takes the closure of the simplex, which means you take the convex hull of the basis vectors and $\vec 0$, so these are not probability measures. The fixed point of a transformation of probability measures may end up being in the closure instead, such as $\vec 0$. $\endgroup$ – Douglas Zare Apr 20 '13 at 9:31

$\begingroup$ Martin Hairer has some good [lecture notes][1] on this sort of thing. [1] hairer.org/notes/Markov.pdf $\endgroup$ – user32372 Apr 20 '13 at 12:14

$\begingroup$ By the way, I think the shift mentioned by Anthony Quas is better if it is on $\mathbb{N}$. On $\mathbb Z$, adding one preserves uniform measures which don't add up to $1$. On $\mathbb N$, the shift only preserves $\vec 0$. $\endgroup$ – Douglas Zare Apr 20 '13 at 19:31