Let $(X_t)_{t \in \mathbb{N}}$ be a discrete-time Markov process on the finite state space $\mathcal{X}$, with transition matrix $T$. Suppose $f: \mathcal{X} \to \mathcal{Y}$ is a (deterministic) function such that $(f(X_t))_{t \in \mathbb{N}}$ is a Markov process on the state space $\mathcal{Y}$. What can we say about $f$ in relation to the transition matrix $T$ of the original chain?

Ideally, I would like an exact characterization of when the derived process is Markov. If $\mathcal{X} = \mathcal{Y} \times \mathcal{Z}$ and $f$ is projection onto the first component, then I think it suffices for the transition probability $T((y_1,z_1),(y_2,z_2))$ to be constant in $z_1$ or constant in $z_2$. There is also the case when $f$ is injective. Are there any other cases?

  • 2
    $\begingroup$ A general criterion when the function of a Markov chain is again a Markov chain is given in the following article: F. P. Kelly, Markovian Functions of a Markov Chain, Sankhyā: The Indian Journal of Statistics, Series A (1961-2002) , Vol. 44, No. 3 (Oct., 1982), pp. 372-379. Link: jstor.org/stable/25050326 $\endgroup$ – Stephan Sturm Jun 29 '13 at 1:12

Are there any other cases?

Sure there are. Say, $(X_t)_{t\in\mathbb{N}}$ is the same as a chain $(Y_t)_{t\in\mathbb{N}}$ except that it has two copies $a_0,a_1$ of a certain state $a$ of $(Y_t)_t$. Every time $(Y_t)_t$ has a transition to $a$, $(X_t)_t$ flips a coin to decide which of $a_0$ or $a_1$ to go. For transitions out of $a$, $(X_t)_t$ simply ignores the distinction between $a_0$ and $a_1$. The function $f$ maps $a_0$ and $a_1$ to $a$ and every other state to itself.

The general criterion on $f$ to ensure that $(f(X_t))_t$ is a Markov chain is that for every $x,x'\in\mathcal{X}$ with $f(x)=f(x')$ and every $y\in\mathcal{Y}$, the transition probabilities should satisfy $T(x,f^{-1}(y))=T(x',f^{-1}(y))$.

A related question: under what conditions the projected process is a $k$-step Markov chain?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.