I have a two state, discrete time, time-inhomogeneous Markov process with transition matrix defined by $$T_i=\begin{pmatrix} 1-p_i\alpha & p_i\alpha \\ p_i\beta& 1-p_i\beta \end{pmatrix}$$ where $p_i$ is a random variable and $\alpha,\beta$ are both defined on $[0,1]$. In the application I have in mind, it would be reasonable to assume that the $p_i$ are iid in a probability distribution on $[0,1)$ or (probably) a small subset.

I have done some numerical experiments in SAGE and I believe that this process has a steady state/equilibrium/stationary distribution (whatever you want to call it), but I am having trouble finding literature on the existence of steady states for time inhomogeneous Markov processes. If it helps, I am reasonably sure that one could also apply the adjectives irreducible, aperiodic, and possibly ergodic to this Markov process.

I have found a theorem that says that a finite-state, irreducible, aperiodic Markov process has a unique stationary distribution (which is equal to its limiting distribution). What is not clear (to me) is whether this theorem is still true in a time-inhomogeneous setting.

Does anyone know of a theorem I could reference that gives the existence of this solution? I am out of my area of expertise, and in this case I would just need the reference. Alternatively, if you know of a more hack-and-slash way to determine if an equilibrium exists in this case (eigenvalues? limits of products of random matrices?), I'm willing to do some computations!

homogeneousMarkov process if the p_i are iid. $\endgroup$ – guest Jun 22 '15 at 20:23reversetime-ordering operator. Now an initial distribution $\pi(0)$ is propagated as $\pi(t) = \pi(0)U(t)$. (See, e.g., Kleinrock, L.Queueing Systems, vol. 1. Wiley (1975).) $\endgroup$ – Steve Huntsman Jun 22 '15 at 21:183more comments