# Birkhoff Ergodic Theorem and Ergodic Decomposition Theorem for Continuous-Time Markov Processes

I have a couple of questions regarding ergodicity for Markov processes in continuous time. (In particular, the first question seems like it should be particularly basic, and yet I haven't managed to find a proof [or counterexample!].)

We have a family $$(P_x^t)_{x \in \mathbb{R},t \geq 0}$$ of Borel probability measures on $$\mathbb{R}$$ such that

1. for all $$A \in \mathcal{B}(\mathbb{R})$$, the map $$(x,t) \mapsto P_x^t(A)$$ is Borel-measurable;

2. for all $$x \in \mathbb{R}$$, $$P_x^0=\delta_x$$;

3. for all $$A \in \mathcal{B}(\mathbb{R})$$ and $$s,t \geq 0$$, $$P_x^{s+t}(A)=\int_\mathbb{R} P_y^t(A) \, P_x^s(dy)$$.

[We can refer to $$(P_x^t)_{x \in \mathbb{R},t \geq 0}$$ as a "measurable stochastic semigroup". In general, "stochastic semigroups" only need to be measurable in $$x$$ for each $$t$$.]

We will say that a probability measure $$\rho$$ on $$\mathbb{R}$$ is stationary if $$\rho(A)=\int_\mathbb{R} P_x^t(A) \, \rho(dx)$$ for all $$A \in \mathcal{B}(\mathbb{R})$$ and $$t \geq 0$$. We will say that a probability measure on $$\mathbb{R}$$ is ergodic if it is an extremal point of the convex set of stationary probability measures.

Q1. Let $$(\Omega,\mathcal{F},(\mathcal{F}_{t \geq 0}),\mathbb{P})$$ be a filtered probability space, and let $$(X_t)_{t \geq 0}$$ be a progressively measurable real-valued homogeneous Markov process with transition probabilities given by $$(P_x^t)_{x \in \mathbb{R},t \geq 0}$$ -- that is to say, $$P_{X_s(\cdot)}^t(A)$$ is a conditional probability of $$X_{s+t}^{-1}(A)$$ with respect to $$\mathcal{F}_s$$ (for all $$s,t,A$$). Suppose also that $$\rho:=X_{0\ast}\mathbb{P}$$ is stationary. Fix any bounded measurable $$f:\mathbb{R} \to \mathbb{R}$$; is it the case that

$$\hspace{5mm} \lim_{T \to \infty} \frac{1}{T} \int_0^T f(X_t(\omega)) \, dt$$

exists for $$\mathbb{P}$$-almost all $$\omega \in \Omega$$?

(Please note that we do not assume any kind of continuity of $$(X_t)$$, but only that it is progressively measurable.)

Now in terms of my motivation, what I am really after is an ergodic decomposition theorem for the setting that I'm currently working with; I think that a positive answer to Q1 will be enough for me to prove this. However, I would ideally like to know if ergodic decompositions exist more generally:

Q2. Suppose $$\rho$$ is a stationary probability measure. Does there exist a probability measure $$Q$$ on the set $$\mathcal{M}$$ of probability measures on $$\mathbb{R}$$ (equipped with the usual $$\sigma$$-algebra, which is known to be standard) such that

1. $$Q$$-almost every $$\mu \in \mathcal{M}$$ is ergodic;

2. for all $$A \in \mathcal{B}(\mathbb{R})$$, $$\rho(A) = \int_\mathcal{M} \mu(A) \, Q(d\mu)$$?

The following might be useful:

Equivalent definitions of ergodicity: Given a stationary probability measure $$\rho$$, we will say that a set $$A \in \mathcal{B}(\mathbb{R})$$ is $$\rho$$-almost stationary if for all $$t \geq 0$$, $$\rho(x \in A: P_x^t(A)=1)=\rho(A)$$.

(1) In analogy to Proposition 7.2.4 of books.google.co.uk/books?isbn=0521515971 (p378) for deterministic systems, we have that a stationary probability measure $$\rho$$ is ergodic if and only if every $$\rho$$-almost stationary set has $$\rho$$-trivial measure: If $$\rho(A) \in (0,1)$$ and $$A$$ is $$\rho$$-almost stationary, then $$\rho$$ conditioned on $$A$$ and $$\rho$$ conditioned on $$\mathbb{R} \setminus A$$ are stationary probability measures which can be linearly combined in the obvious way to give $$\rho$$. In the other direction, it suffices to show that if every $$\rho$$-almost stationary set has trivial measure and $$\tilde{\rho}$$ is a stationary probability measure that is absolutely continuous with respect to $$\rho$$, then $$\rho=\tilde{\rho}$$. Take a density $$g$$ of $$\tilde{\rho}$$ with respect to $$\rho$$. For each $$t$$, define the probability measure $$\rho_t$$ on $$\mathbb{R} \times \mathbb{R}$$ by $$\rho_t(A \times B) = \int_A P_x^t(B) \, \rho(dx)$$. The stationarity of $$\tilde{\rho}$$ implies that

$$\hspace{5mm} \int_{A \times (X \setminus A)} g(x_1) \, \rho_t(d(x_1,x_2)) \ = \ \int_{(X \setminus A) \times A} g(x_1) \, \rho_t(d(x_1,x_2))$$

for any $$A \in \mathcal{B}(\mathbb{R})$$ and $$t \geq 0$$. Setting $$A:=\{x \in X : g(x) \geq 1\}$$, the above equation (combined with the stationarity of $$\rho$$) implies that $$A$$ is $$\rho$$-almost stationary, so $$A$$ has trivial measure. It follows that $$\tilde{\rho}=\rho$$.

(2) We will say that a set $$A \in \mathcal{B}(\mathbb{R})$$ is invariant if for all $$t \geq 0$$ and all $$x \in A$$, $$P_x^t(A)=1$$. Given a set $$A$$ that is $$\rho$$-almost stationary, there exists a set $$A'$$ that is invariant, with $$\rho(A \triangle A')=0$$. Namely, set

$$\hspace{5mm} A' \ := \ \{ x \in X : \textrm{Leb}(t \geq 0 : P_x^t(A)<1) = 0 \}$$

where $$\textrm{Leb}$$ denotes the Lebesgue measure. So a stationary probability measure $$\rho$$ is ergodic if and only if every invariant set has $$\rho$$-trivial measure.

(It is perhaps worth pointing out that (1) does not rely on the stochastic semigroup $$(P_x^t)$$ being a "measurable" stochastic semigroup, but the construction in (2) does rely on this.)

Update: I'm pretty sure the answer to Q2 is yes, because I think I can prove it using an ergodic theorem for measurable stochastic semigroups; namely, letting $$\rho$$ be a stationary probability measure, I think I can first prove that for any bounded measurable $$f:\mathbb{R} \to \mathbb{R}$$,

$$\hspace{5mm} \lim_{T \to \infty} \frac{1}{T}\int_0^T \int_\mathbb{R} \! f(y) P_x^t(dy) \; dt$$

exists for $$\rho$$-almost all $$x \in \mathbb{R}$$, with the limit (as a function of $$x$$) being a conditional expectation of $$f$$ over the probability space $$(\mathbb{R},\mathcal{B}(\mathbb{R}),\rho)$$ with respect to the $$\sigma$$-algebra of $$\rho$$-almost stationary sets. (As mentioned in Kifer's book "Ergodic Theory of Random Transformations", the discrete-time analogue of the above statement can be obtained as a special case of the Chacon-Ornstein ergodic theorem.) Using this fact, it should be possible to prove the ergodic decomposition theorem (by a similar approach as in the proof for deterministic dynamical systems).

However, I suspect that the answer to Q1 is no (although I do not have a counterexample!!). More precisely, I suspect that the answer to Q1 is the same as the answer to my question Is it true that all stationary measurable stochastic processes are "measurably stationary"? - and I expect that the answer to that question is no (although again, I do not have a counterexample).

If the answer to Q1 is no, I wonder whether perhaps it becomes yes in the particular case that $$(X_t)_{t \geq 0}$$ is a strong Markov process.

I have the answers to my two questions. (I've actually had them for a while; apologies for the delay in posting.) I will give them in reverse order:

The answer to Q2 is yes; the structure of the proof is exactly as I outlined in the update. Details can be found in section 5 (in particular, Corollary 100) of my notes http://wwwf.imperial.ac.uk/~jmn07/Ergodic_Theory_for_Semigroups_of_Markov_Kernels.pdf.

The answer to Q1 is also yes: Since $f$ is bounded, it is sufficient just to consider the limit as $T$ tends to $\infty$ in the integers. By the positive answer to the question Is it true that all stationary measurable stochastic processes are "measurably stationary"?, the discrete-time stochastic process $\left(\int_n^{n+1} f(X_t(\cdot)) \, dt \right)_{n \geq 0}$ is stationary, and therefore Birkhoff's ergodic theorem (applied to the shift map on $\mathbb{R}^{\mathbb{N}_0}$) gives the desired convergence.

• However, I realise that what I have asked for in Q1 is weaker than what I should be asking for - see my newer question mathoverflow.net/questions/199540/…. Mar 10, 2015 at 3:35

As you pointed out correctly ergodic measures are extremes of the convex set formed by invariant (w.r.t. to the semigroup) measures. As such, if you can show uniqueness of invariant measure then the result you are looking for will follow immediately by Birkhoff's ergodic theorem as you point out.

Uniqueness of invariant measure for Markov processes usually follows by Harris theorem ( see for example, "Yet another look at Harris' ergodic theorem for Markov chains" by Martin Hairer and Jonathan Mattingly ) which depends on specific properties of the semi-group and not necessarily on the path wise properties of the process itself.

• Sorry for the late response. Thank you very much for the above information about unique ergodicity, I hadn't been aware of this theorem, and it looks quite nice. Now it seems that you may have perhaps misunderstood my question, seeing as you said that in the case of unique ergodicity, the "result I am looking for" follows by Birkhoff's ergodic theorem. If I already have unique ergodicity then ergodic decomposition is a tautology and has nothing to do with Birkhoff's ergodic theorem; the whole point of ergodic decomposition is for cases where I might not have unique ergodicity. Sep 23, 2014 at 14:24