All Questions

Consider a discrete time Markov chain $(X_t)$ on a finite state space $\mathcal{S}$, with transition matrix $P$. Assume that the chain admits a stationary distribution $\pi$, which I will identify ...

By processes, I mean discrete, stationary stochastic processes, that is $(X,\mathcal{U},\mu,T)$ where $X$ is the set of doubly infinite sequences of some alphabet $A$, $\mathcal{U}$ is the $\sigma$-...

This same question was also posted on MSE https://math.stackexchange.com/questions/3327007/existence-and-uniqueness-of-a-stationary-measure. Recently I have posted the following question on MO ...

I have a Markov Chain of $N$ states. Such states represent the energy levels in a molecule. The states' connectivity is as follows: States $j\in\{0,\ldots,N\}$ transition to $k\in\{\max(j-M,0),...,\...

Consider $X_n\in \mathcal{X}$ a controlled Markov chain taking value in a compact set $\mathcal{X}$ with action $a\in \mathcal{A}$, where the action set $|\mathcal{A}|$ is finite. (In particular, we ...