Difference in probability distributions from two different kernels

Question

Let $(E,\mathscr E)$ be a measurable space and $P,\tilde P$ be two stochastic kernels on that space. I wonder how the induced measures $\mathsf P_x$ and $\tilde{\mathsf P}_x$ differ on the space of finite trajectories $(\Omega_n,\mathscr F_n)$ and on the space of infinite trajectories $(\Omega,\mathscr F)$. I've already asked the finite-trajectories part of the question here and get two answers proving that $$ \sup\limits_{x\in E}\sup\limits_{C\in \mathscr F_n}|\tilde{\mathsf P}_x(C) - \mathsf P_x(C)| \leq n\|\tilde P - P\|. \tag{1} $$

1. Unfortunately, I didn't see in replies any canonical references on $(1)$ so I wonder if somebody has seen anything that looks like that result - or anything on the topic I've described.

2. I also wonder, when the rhs in $(1)$ can be bounded if we take the supremum over $C\in \mathscr F$, since clearly $(1)$ does not help in that case. I know that in general there are no trivial bounds since: there exists the example of the kernel $P$ and the pair $x,C$ such that $\mathsf P_x(C) = 1$ but for any $\delta>0$ there is $\tilde P^\delta$ such that $\|\tilde P^\delta - P\|\leq \delta$ but $\tilde{\mathsf P}_y(C) = 0$ for all $y$. On the other hand, I thought that some structural assumptions on $P$ may help: e.g. uniqueness of the invariant distribution and convergence of $P^n$ to this distribution. Again, I would be highly interested in references on the topic of how $(1)$ can be extended from $\mathscr F_n$ to $\mathscr F$, if there are any.

---

To be precise, I give here the formal statement of the problem.

Let $(E,\mathscr E)$ be a measurable space and for each $n\geq 0$ define $(\Omega_n,\mathscr F_n) = (E^{n+1},\mathscr E^{\otimes(n+1)})$ to be a measuruable product space. In the same fashion, let $\Omega = E^{\mathbb N_0}$ and $\mathscr F$ be its product $\sigma$-algebra.

Let $P$ be a stochastic kernel on $(E,\mathscr E)$, i.e. $P(x,\cdot)$ is a probability measure for each $x\in E$ and $P(\cdot,A)$ is a measurable function for each $A\in \mathscr E$. There is the unique family of measures $(\mathsf P_x)_{x\in E}$ on the space $(\Omega,\mathscr F)$ such that $$ \mathsf P_x(A_0\times A_1\times\dots \times A_n) = 1_{A_0}(x)\int\limits_{A_1}\dots\int\limits_{A_n}P(x_{n-1},\mathrm dx_n)\dots P(x,\mathrm dx_1) \tag{2} $$ where $n\geq0$ is any and $A_i\in \mathscr E$. Moreover, for each $n\geq 0$ let us denote $(\mathsf P^n_x)_{x\in E}$ to be the family of measures on the space $(\Omega_n,\mathscr F_n)$ which is defined uniquely by $(2)$.

Finally, the norm $\|\tilde P - P\|$ is defined by $$ \|\tilde P - P\| = \sup\limits_{f\in \mathrm b\mathscr E}\frac{\|\tilde Pf - Pf\|}{\|f\|} $$ where $\mathrm b\mathscr E$ is a Banach space of real-value bounded measurable functions with a norm $\|f\| = \sup\limits_{x\in E}|f(x)|$ and $$ Pf(x) = \int\limits_E f(y)P(x,\mathrm dy). $$

Answer 1

It's much easier to understand and answer this question by using the (equivalent) language of families of transition probabilities corresponding to stochastic kernels. So, let $(\pi_x)$ (resp., $(\tilde\pi_x)$ be the family of transition probabilities corresponding to the kernel $P$ (resp., $\tilde P$). Then $$\|P-\tilde P\|=\sup_x \|\pi_x-\tilde\pi_x\|$$ (note that in the RHS I am using the definition of the total variation, according to which the distance between two mutually singular measures equals 2 - it's twice the distance which you use in formula (1)).

Let $\mathbf P^n_x$ and $\widetilde{\mathbf P^n_x}$ be the corresponding measures on the space of sample paths from time $0$ to time $n\le \infty$, and put $$\|P-\tilde P\|^{(n)}=\sup_x \|\mathbf P^n_x-\widetilde{\mathbf P^n_x}\|,$$ so that $\|P-\tilde P\|^{(1)}=\|P-\tilde P\|$. You are interested in estimating $\|P-\tilde P\|^{(n)}$ in terms of $\|P-\tilde P\|$.

By the definition of the total variation, the "overlap" of two probability measures $\mu,\tilde\mu$ is $$\|\mu\wedge\tilde\mu\|=1-\|\mu-\tilde\mu\|/2$$ Since the measure $\mathbf P^{n+1}_x$ naturally projects onto the measure $\mathbf P^n_x$, and the conditional measures of this projection are precisely the one-step transition probabilities, the numbers $$\phi_n = \inf_x \| \mathbf P^n_x\wedge\widetilde{\mathbf P^n_x} \| = 1-\|P-\tilde P\|^{(n)}/2$$ satisfy the inequality $$ \phi_{n+1} \ge \phi_n \phi_1, $$ and one arrives precisely at George Lowther's inequality. Moreover, one can easily see that it is optimal.

I think this is as much as what one can say about your question in this generality. Two more comments:

1. Given a big finite state space $X$, the number of parameters describing probability distributions on $X$ is much less than the number of parameters describing Markov chains on $X$. The moral is that there is a lot of of pairs of distinct irreducible Markov chains with the same stationary distributions. The corresponding measures on the infinite path spaces of these chains will obviously be mutually singular. For the simplest example one can take a Bernoulli measure and any non-Bernoulli Markov chain with the same stationary distribution (such an example exists even for $|X|=2$).

2. There is a general Kakutani theorem which gives necessary and sufficient conditions for two measures on an infinite product space to be equivalent in terms of their finite dimensional distributions.

EDIT: I will answer here as comments don't provide enough space.

(1) Yes, $\pi_x=P(x,\cdot)$.

(2a) Let $\lambda_+$ and $\lambda_-$ be, respectively, the positive and the negative parts of the difference $\lambda=\mu-\tilde\mu$, then the total variation distance between $\mu$ and $\tilde\mu$ is $$ \|\mu-\tilde\mu\| = \|\lambda_+\| + \|\lambda_-\| = 2 \|\lambda_+\| = 2 \|\lambda_-\| $$ (by the way, this is the reason why it is this quantity that should be called the "total variation distance", not its half). The "common part" of the measures $\mu$ and $\tilde\mu$ is $$ \mu\wedge\tilde\mu=\mu-\lambda_+=\tilde\mu-\lambda_- $$ (this is the infimum of $\mu$ and $\tilde\mu$ with respect to the natural order on the cone of positive measures).

(2b) In this situation the projection business is very simple as one deals with product spaces. If you have a measure $m$ on a space $X$ and a family of measures $m_x$ on another space $Y$ parametrized by points of $X$, then the result of integration of the family $m_x$ against $m$ gives you a new measure on the product space $X\times Y$. One can consider this construction as a generalization of the Fubini theorem (actually, under reasonable conditions on the involved measures the converse is also true, namely, any measure on $X\times Y$ can be uniquely presented in this way - this is a particular case of the famous theorem on existence of conditional measures). It is precisely this construction that is used when passing from $\mathbf P^n_x$ to $\mathbf P^{n+1}_x$.

(3) Actually what I meant is not the original Kakutani theorem, but its generalization to arbitrary pairs of locally equivalent measures - see, for instance, the textbook of Shiryaev, Section VII.6. However, it would only help in your situation if you deal with Markov chains without finite stationary measures. The reason is the following. For a finite stationary measure of a Markov chain the associated shift invariant measure is ergodic if and only if the state space does not admit a decomposition into two non-communicating subsets of positive measure. On the other hand, any two distinct ergodic measures are mutually singular, so that $\mathbf P^\infty_x$ and $\widetilde{\mathbf P^\infty_x}$ are also pairwise singular for a.e. $x$.

EDIT 2: Once again, hanc marginis exiguitas non caperet, so that I am adding another edit to the original answer.

1. Yes, when talking about conditional decompositions of measures on product spaces I meant that any probability measure $M$ on $X\times Y$ can be uniquely decomposed as $dM(x,y)=dm(x)dm_x(y)$, or, equivalently, $M=\int m_x dm(x)$, where $m$ is its projection onto $X$, and $m_x$ are its conditional measures on the fibers of this projection. In your case $$d \mathbf P_x^{n+1}(x,x_1,\dots,x_n,x_{n+1})= d \mathbf P_x^n(x,x_1,\dots,x_n)d\pi_{x_n}(x_{n+1}).$$ [Another general comment: there are no sets in my formulas for measures - this notation is actually much more convenient.]

2. Let me explain it in the general setup of measures on product spaces. Let $M$ and $\tilde M$ be two measures on the same product space $X\times Y$ with conditional decompositions $M=\int m_x dm(x)$ and $\tilde M=\int \tilde m_x d\tilde m(x)$. Then $$\|M\wedge \tilde M\| = \int \|m_x\wedge\tilde m_x\| d (m\wedge\tilde m)(x),$$ whence $$\|M\wedge \tilde M\| \ge \|m\wedge\tilde m\| \inf_x \|m_x\wedge\tilde m_x\|,$$ which in your particular case takes precisely the form $$ \phi_{n+1} \ge \phi_n \phi_1 . $$