Absolutely continuous coupling of probability measures

Question

I have a Borel probability measure $\pi$ on $\mathbb{R}^{n+1}$ such that $\pi_1=\mu_1, \ldots, \pi_{n+1}=\mu_{n+1}$ for some fixed Borel probability measures $\mu_1, \ldots, \mu_{n+1}$ (where each $\mu_i$ is absolutely continuous with respect to Lebesgue measure). I want to construct a sequence of probability measures $\pi^{(k)}$ such that $\pi^{(k)}$ converges weakly to $\pi$ and has the same marginals as $\pi,$ that is, $\pi^{(k)}_j=\mu_j$ for $j=2,\ldots, n+1.$

Now of course, I can take $\pi^{(k)}=\pi.$ But, I am trying to prove some inequality where I need $\pi^{(k)}$ to be absolutely continuous with respect to Lebesgue measure (on $\mathbb{R}^n$).

Now I have a twofold goal: I can make $\pi^{(k)}$ absolutely continuous but then in the process I end up messing with the marginals. Can someone point it out to me if it is possible at all to cook up such a sequence $\pi^{(k)}$ which is absolutely continuous with respect to Leb, and has the fixed marginals and which weakly converges to $\pi.$

Answer 1

Let me formulate and prove it in greater generality (which actually makes your question easier). Let $X$ be a metric space, and $\mu$ be a probability measure on $X\times X$ (for simplicity I consider the product of two copies of $X$ only; the general case is precisely the same). You want to obtain a sequence of measures $\nu_n$ on $X\times X$ which

(1) have the same marginals $\mu_1,\mu_2$ as $\mu$;

(2) are absolutely continuous with respect to the product measure $\mu_1\times\mu_2$;

(3) weakly converge to $\mu$.

For each $n$ take a countable partition of $X$ into measurable sets $X^i$ with diameter $\le 1/n$ (presuming the space $X$ is such that partitions like this exist for any $n$) and denote by $\mu_\epsilon^i$ the normalized restriction of the measure $\mu_\epsilon$ to $X^i$. Then put $$ \nu_n = \sum_{i,j} \mu(X^i\times X^j) \mu_1^i\times \mu_2^j \;. $$

Answer 2

There are various papers where this question occurs. I guess a paper which directly covers the case you are interested in is https://arxiv.org/pdf/1901.07407.pdf . Note that here, the marginals don't have to be one dimensional.

A more general case in polish spaces is studied in https://arxiv.org/pdf/1811.00304.pdf Proposition 2. Since there is no Lebesgue measure on polish spaces, here the question is about absolute continuity with respect to the product measure of the marginals. Since in your case the product measure is absolutely continuous with respect to the Lebesgue measure, this of course also answers your problem.

There are other papers which incorporate this problem, mostly in the context of regularized optimal transport, but I do not have the references at hand.

Note also that in your case, with one dimensional marginals, things are simple as the concept of quantile functions and copulas can be used (similar in spirit to Iosif's answer) .

Answer 3

$\newcommand{\R}{\mathbb R}$ Here is an idea of the desired construction. You have a Borel measure $\pi$ on $\R^n$ with marginals $\mu_1,\dots,\mu_n$. Let $X:=(X_1,\dots,X_n)$ be a random point in $\R^n$ with distribution $\pi$. For each $j=1,\dots,n$, let $U_j:=F_j(X_j)$, where $F_j$ is the cumulative distribution function of $X_j$, so that $U_j$ is uniformly distributed on the interval $(0,1)$, which we will identify with the additive group $G:=\R/\mathbb Z$. So, $U:=(U_1,\dots,U_n)$ is a random element of the group $G^n$ with uniform marginals. Let $V:=(V_1,\dots,V_n)$ be an independent copy of $U$. For each natural $k$, let $W_k=(W_{k,1},\dots,W_{k,n}):=U\oplus(V/k)$, where $\oplus$ denotes the addition in the group $G^n$, so that $W_k$ is a random element of the group $G^n$ with an absolutely continuous distribution and uniform marginals.

Finally, let $\pi_k$ be the distribution of the random point $X_k:=(X_{k,1},\dots,X_{k,n})$ in $\R^n$, where $X_{k,j}:=F_j^{-1}(W_{k,j})$ and $$F_j^{-1}(u):=\inf\{x\in\R\colon F_j(x)\ge u\}=\min\{x\in\R\colon F_j(x)\ge u\} $$ for $u\in(0,1)$. Then for each $k$ the probability measure $\pi_k$ will be absolutely continuous with marginals $\mu_1,\dots,\mu_n$, and $\pi_k$ will weakly converge to $\pi$ as $k\to\infty$.

Answer 4

If you work on compact domains instead of $\RR$ (or if you assume that all marginals have compact support), you can use couplings from regularized optimal transport: For $n=1$ (i.e. a coupling of just two marginals) there are two popular ways.

Entropic regularization: Use $\pi_k$ defined by $$\newcommand{\RR}{\mathbb{R}} \min_\pi \int_{\RR^2} cd\pi + \gamma_k\int_{\RR^2}\pi(\log(\pi)-1)d\lambda $$ where $\lambda$ denotes the Lebesgue measure, $c$ is some cost function (e.g. continuous and non-negative), and $\gamma_k$ is a sequence of regularization parameters converging to zero and the minimum is taken over all measures $\pi$ which have a density with respect to the Lebesgue-measure (and the density has finite entropy), which have their support in the product of the supports of the marginals, and which have the marginals $\mu_1$ and $\mu_2$. If $\gamma_k$ tends to zero it should hold that $\pi_k$ converges weakly (in the sense of measures) to $\pi$.

Quadratic regularization: Use $\pi_k$ defined by $$\newcommand{\RR}{\mathbb{R}} \min_\pi \int_{\RR^2} c d\pi + \gamma_k\int_{\RR^2}\pi^2 d\lambda $$ where $\lambda$ denotes the Lebesgue measure, $c$ is again some cost function (e.g. square integrable and non-negative) and $\gamma_k$ is a sequence of regularization parameters converging to zero and the minimum is taken over all measures $\pi$ which have a square integrable density with respect to the Lebesgue-measure, having their support in the product of the supports of the marginals and which have the marginals $\mu_1$ and $\mu_2$. I guess that weak convergence is also true here.

For entropic regularization I recommend Computational Optimal Transport (free version here) and for quadratic regularization I only have my own paper Quadratically Regularized Optimal Transport (also on the arXiv).