Let $S$ be a finite set of states. Let $(M_n)$ be a sequence of transitions on $S$; that is, for every natural number $n$, $M_n$ is a non-negative $|S| \times |S|$ matrix whose rows sum up to 1. Suppose that the limit $M :=\lim_{n \to \infty} M_n$ exists (and is a transition matrix; here limit means that for every entry $(i, j)$, the sequence $(M_n(i,j))_n$ converges). I am interested in the Cesaro mean $\lim_{N \to\infty} \frac{1}{N}\sum_{n=1}^N \left(\prod_{i=1}^nM_i\right)$: under which conditions on $(M_n)$ does the Cesaro mean exist? Plainly, if $(M_n)$ converges quickly to $M$ (for example, $\sum_{n=1}^\infty \|M_n- M\| < \infty$) then the Cesaro mean exists. Another trivial case in which the Cesaro mean exists is when there is only one stable set under $M$. There is literature when the rank of the limit $M$ is 1. I am interested in the case where $M$ is general.

A more general question concerns the case when $(M_n)$ is a stochastic process: the transition at stage $n$ depends on past realizations (and maybe also on other random factors).

Eilon