By processes, I mean discrete, stationary stochastic processes, that is $(X,\mathcal{U},\mu,T)$ where $X$ is the set of doubly infinite sequences of some alphabet $A$, $\mathcal{U}$ is the $\sigma$-algebra generated by the coordinates, $\mu$ is a probability measure on $(X,\mathcal{U})$, and $T$ is the left shift by one.

Finitarily Markovovian processes (as defined by Morvai and Weiss in On Estimating the Memory for Finitarily Markovian Processes) are those processes $\{X_n\}_{n=-\infty}^{\infty}$ for which there is a finite $K$ $(K=K(\{X_n\}_{n=-\infty}^0))$ such that the conditional distribution of $X_1$ given the entire past is equal to the conditional distribution of $X_1$ given only $\{X_n\}_{n=1-K}^0$.

First, can anyone suggest good literature on the topic of when a hidden Markov process is finitarily Markov?

Specifically, I would like to know when a finite factor (also called $k$-block factor or simply block factor) of a Bernoulli scheme is finitarily Markovian?

The following is an example of a 2 block factor of a Bernoulli scheme which is not finitarily Markovian.

Let $X=\{0,1\}^{\mathbb{Z}}$ with measure $\mu=(1/3,2/3)^{\mathbb{Z}}$. Define a map $\phi:X \to Y$ where $(\phi(x))_i=(x_i+x_{i+1})$mod $2$.

2more comments