Defining measures through products of Markov kernels

Question

I am quite puzzled by the expression given in equation 21 (page 10) in this paper, https://arxiv.org/pdf/1802.09188.pdf

Its LHS seems to be a measure $\nu_n^N$ and hence I guess it takes as argument measurable sets. Equation 21 is defining the measure $\nu_n^N$.

But its RHS is weighted sum of products of a measure $\mu_0$ and Markov kernels $R_{\gamma_i}$ As defined in its own equation 12, $R_\gamma$s are standard Markov kernels which needs 2 arguments a point and a measurable set.

• So how is one to read this equation?

• Is there a context to these kinds of expressions, that I am missing? Like, is this a familiar construction in some scenario? It would be great to get some pedagogic reference as to from where has this come!

Answer 1

For a measure $\mu$ and a Markov kernel $R$ on a measurable space $X$, it's standard to use $\mu R$ to denote the measure defined by $$(\mu R)(A) = \int_X R(x, A) \mu(dx).$$ If you think of $R$ as the one-step transition function of a Markov chain, then $\mu R$ gives the distribution of the process after one step if its initial distribution was $\mu$: $(\mu R)(A) = P_\mu(X_1 \in A)$.

This lets you view $R$ as an operator on the space $\mathcal{P}(X)$ of probability measures on $X$, acting on the right (see below), and you can "multiply" such operators by composing them. Thus for Markov kernels $R_1, R_2$, we define $Q = R_1 R_2$ by $$(\mu Q)(A) = ((\mu R_1)R_2)(A) = \int_X R_2(x,A) (\mu R_1)(dx) = \int_X \int_X R_2(y,A) R_1(x, dy) \mu(dx).$$ So if $R_1, R_2$ are the 1-step transition functions for an inhomogeneous Markov chain from times 0 to 1 and 1 to 2 respectively, then $Q$ gives the 2-step transition function from time 0 to 2: $(\mu Q)(A) = P_\mu(X_2 \in A)$. Indeed, you can also check that $Q$ corresponds to the Markov kernel defined by $$Q(y,A) = \int_X R_2(x,A) R_1(y, dx)$$ so that this convolution-like operation defines a natural multiplication on Markov kernels themselves.

And you define products of three or more such operators by iterating this. You can check that this multiplication is associative (though not commutative), so $R_1 R_2 \dots R_k$ is unambiguous.

The reason for the "right action" notation comes from the fact that $R$ can also act on the space $B(X)$ of bounded measurable functions on $X$ via $(Rf)(x) = \int_X f(y) R(x, dy)$, and there is the natural dual pairing between $\mathcal{P}(X)$ and $B(X)$ given by integration, i.e. $\mu f = \int_X f\,d\mu$. In this notation everything turns out to be associative, e.g. $(\mu R)f = \mu(Rf)$. For similar reasons, some authors write the action of $R$ on $\mathcal{P}(X)$ as $R^* \mu$ instead of $\mu R$, thinking of this as the adjoint of $R$'s left action on $B(X)$.

You already get a glimpse of this in the finite-state case, where it is common to view $R$ as a square matrix, $f$ as a column vector, and $\mu$ as a row vector. Then all the notations $\mu f$, $\mu R$, $R f$, etc, correspond to matrix multiplication with all the dimensions matching up.