# Birkhoff Ergodic Theorem and Ergodic Decomposition Theorem for Continuous-Time Markov Processes

I have a couple of questions regarding ergodicity for Markov processes in continuous time. (In particular, the first question seems like it should be particularly basic, and yet I haven't managed to find a proof [or counterexample!].)

We have a family $$(P_x^t)_{x \in \mathbb{R},t \geq 0}$$ of Borel probability measures on $$\mathbb{R}$$ such that

1. for all $$A \in \mathcal{B}(\mathbb{R})$$, the map $$(x,t) \mapsto P_x^t(A)$$ is Borel-measurable;

2. for all $$x \in \mathbb{R}$$, $$P_x^0=\delta_x$$;

3. for all $$A \in \mathcal{B}(\mathbb{R})$$ and $$s,t \geq 0$$, $$P_x^{s+t}(A)=\int_\mathbb{R} P_y^t(A) \, P_x^s(dy)$$.

[We can refer to $$(P_x^t)_{x \in \mathbb{R},t \geq 0}$$ as a "measurable stochastic semigroup". In general, "stochastic semigroups" only need to be measurable in $$x$$ for each $$t$$.]

We will say that a probability measure $$\rho$$ on $$\mathbb{R}$$ is stationary if $$\rho(A)=\int_\mathbb{R} P_x^t(A) \, \rho(dx)$$ for all $$A \in \mathcal{B}(\mathbb{R})$$ and $$t \geq 0$$. We will say that a probability measure on $$\mathbb{R}$$ is ergodic if it is an extremal point of the convex set of stationary probability measures.

Q1. Let $$(\Omega,\mathcal{F},(\mathcal{F}_{t \geq 0}),\mathbb{P})$$ be a filtered probability space, and let $$(X_t)_{t \geq 0}$$ be a progressively measurable real-valued homogeneous Markov process with transition probabilities given by $$(P_x^t)_{x \in \mathbb{R},t \geq 0}$$ -- that is to say, $$P_{X_s(\cdot)}^t(A)$$ is a conditional probability of $$X_{s+t}^{-1}(A)$$ with respect to $$\mathcal{F}_s$$ (for all $$s,t,A$$). Suppose also that $$\rho:=X_{0\ast}\mathbb{P}$$ is stationary. Fix any bounded measurable $$f:\mathbb{R} \to \mathbb{R}$$; is it the case that

$$\hspace{5mm} \lim_{T \to \infty} \frac{1}{T} \int_0^T f(X_t(\omega)) \, dt$$

exists for $$\mathbb{P}$$-almost all $$\omega \in \Omega$$?

(Please note that we do not assume any kind of continuity of $$(X_t)$$, but only that it is progressively measurable.)

Now in terms of my motivation, what I am really after is an ergodic decomposition theorem for the setting that I'm currently working with; I think that a positive answer to Q1 will be enough for me to prove this. However, I would ideally like to know if ergodic decompositions exist more generally:

Q2. Suppose $$\rho$$ is a stationary probability measure. Does there exist a probability measure $$Q$$ on the set $$\mathcal{M}$$ of probability measures on $$\mathbb{R}$$ (equipped with the usual $$\sigma$$-algebra, which is known to be standard) such that

1. $$Q$$-almost every $$\mu \in \mathcal{M}$$ is ergodic;

2. for all $$A \in \mathcal{B}(\mathbb{R})$$, $$\rho(A) = \int_\mathcal{M} \mu(A) \, Q(d\mu)$$?

The following might be useful:

Equivalent definitions of ergodicity: Given a stationary probability measure $$\rho$$, we will say that a set $$A \in \mathcal{B}(\mathbb{R})$$ is $$\rho$$-almost stationary if for all $$t \geq 0$$, $$\rho(x \in A: P_x^t(A)=1)=\rho(A)$$.

(1) In analogy to Proposition 7.2.4 of books.google.co.uk/books?isbn=0521515971 (p378) for deterministic systems, we have that a stationary probability measure $$\rho$$ is ergodic if and only if every $$\rho$$-almost stationary set has $$\rho$$-trivial measure: If $$\rho(A) \in (0,1)$$ and $$A$$ is $$\rho$$-almost stationary, then $$\rho$$ conditioned on $$A$$ and $$\rho$$ conditioned on $$\mathbb{R} \setminus A$$ are stationary probability measures which can be linearly combined in the obvious way to give $$\rho$$. In the other direction, it suffices to show that if every $$\rho$$-almost stationary set has trivial measure and $$\tilde{\rho}$$ is a stationary probability measure that is absolutely continuous with respect to $$\rho$$, then $$\rho=\tilde{\rho}$$. Take a density $$g$$ of $$\tilde{\rho}$$ with respect to $$\rho$$. For each $$t$$, define the probability measure $$\rho_t$$ on $$\mathbb{R} \times \mathbb{R}$$ by $$\rho_t(A \times B) = \int_A P_x^t(B) \, \rho(dx)$$. The stationarity of $$\tilde{\rho}$$ implies that

$$\hspace{5mm} \int_{A \times (X \setminus A)} g(x_1) \, \rho_t(d(x_1,x_2)) \ = \ \int_{(X \setminus A) \times A} g(x_1) \, \rho_t(d(x_1,x_2))$$

for any $$A \in \mathcal{B}(\mathbb{R})$$ and $$t \geq 0$$. Setting $$A:=\{x \in X : g(x) \geq 1\}$$, the above equation (combined with the stationarity of $$\rho$$) implies that $$A$$ is $$\rho$$-almost stationary, so $$A$$ has trivial measure. It follows that $$\tilde{\rho}=\rho$$.

(2) We will say that a set $$A \in \mathcal{B}(\mathbb{R})$$ is invariant if for all $$t \geq 0$$ and all $$x \in A$$, $$P_x^t(A)=1$$. Given a set $$A$$ that is $$\rho$$-almost stationary, there exists a set $$A'$$ that is invariant, with $$\rho(A \triangle A')=0$$. Namely, set

$$\hspace{5mm} A' \ := \ \{ x \in X : \textrm{Leb}(t \geq 0 : P_x^t(A)<1) = 0 \}$$

where $$\textrm{Leb}$$ denotes the Lebesgue measure. So a stationary probability measure $$\rho$$ is ergodic if and only if every invariant set has $$\rho$$-trivial measure.

(It is perhaps worth pointing out that (1) does not rely on the stochastic semigroup $$(P_x^t)$$ being a "measurable" stochastic semigroup, but the construction in (2) does rely on this.)

Update: I'm pretty sure the answer to Q2 is yes, because I think I can prove it using an ergodic theorem for measurable stochastic semigroups; namely, letting $$\rho$$ be a stationary probability measure, I think I can first prove that for any bounded measurable $$f:\mathbb{R} \to \mathbb{R}$$,

$$\hspace{5mm} \lim_{T \to \infty} \frac{1}{T}\int_0^T \int_\mathbb{R} \! f(y) P_x^t(dy) \; dt$$

exists for $$\rho$$-almost all $$x \in \mathbb{R}$$, with the limit (as a function of $$x$$) being a conditional expectation of $$f$$ over the probability space $$(\mathbb{R},\mathcal{B}(\mathbb{R}),\rho)$$ with respect to the $$\sigma$$-algebra of $$\rho$$-almost stationary sets. (As mentioned in Kifer's book "Ergodic Theory of Random Transformations", the discrete-time analogue of the above statement can be obtained as a special case of the Chacon-Ornstein ergodic theorem.) Using this fact, it should be possible to prove the ergodic decomposition theorem (by a similar approach as in the proof for deterministic dynamical systems).

However, I suspect that the answer to Q1 is no (although I do not have a counterexample!!). More precisely, I suspect that the answer to Q1 is the same as the answer to my question Is it true that all stationary measurable stochastic processes are "measurably stationary"? - and I expect that the answer to that question is no (although again, I do not have a counterexample).

If the answer to Q1 is no, I wonder whether perhaps it becomes yes in the particular case that $$(X_t)_{t \geq 0}$$ is a strong Markov process.

The answer to Q1 is also yes: Since $f$ is bounded, it is sufficient just to consider the limit as $T$ tends to $\infty$ in the integers. By the positive answer to the question Is it true that all stationary measurable stochastic processes are "measurably stationary"?, the discrete-time stochastic process $\left(\int_n^{n+1} f(X_t(\cdot)) \, dt \right)_{n \geq 0}$ is stationary, and therefore Birkhoff's ergodic theorem (applied to the shift map on $\mathbb{R}^{\mathbb{N}_0}$) gives the desired convergence.