I have a Borel probability measure $\pi$ on $\mathbb{R}^{n+1}$ such that $\pi_1=\mu_1, \ldots, \pi_{n+1}=\mu_{n+1}$ for some fixed Borel probability measures $\mu_1, \ldots, \mu_{n+1}$ (where each $\mu_i$ is absolutely continuous with respect to Lebesgue measure). I want to construct a sequence of probability measures $\pi^{(k)}$ such that $\pi^{(k)}$ converges weakly to $\pi$ and has the same marginals as $\pi,$ that is, $\pi^{(k)}_j=\mu_j$ for $j=2,\ldots, n+1.$

Now of course, I can take $\pi^{(k)}=\pi.$ But, I am trying to prove some inequality where I need $\pi^{(k)}$ to be absolutely continuous with respect to Lebesgue measure (on $\mathbb{R}^n$).

Now I have a twofold goal: I can make $\pi^{(k)}$ absolutely continuous but then in the process I end up messing with the marginals. Can someone point it out to me if it is possible at all to cook up such a sequence $\pi^{(k)}$ which is absolutely continuous with respect to Leb, and has the fixed marginals and which weakly converges to $\pi.$

  • $\begingroup$ All the answers below are pretty good, but unfortunately I can accept only one answer. I am taking some time to go through all the references I have got in the answer before I accept one answer as the final. $\endgroup$ Jan 8 '20 at 8:49

Let me formulate and prove it in greater generality (which actually makes your question easier). Let $X$ be a metric space, and $\mu$ be a probability measure on $X\times X$ (for simplicity I consider the product of two copies of $X$ only; the general case is precisely the same). You want to obtain a sequence of measures $\nu_n$ on $X\times X$ which

(1) have the same marginals $\mu_1,\mu_2$ as $\mu$;

(2) are absolutely continuous with respect to the product measure $\mu_1\times\mu_2$;

(3) weakly converge to $\mu$.

For each $n$ take a countable partition of $X$ into measurable sets $X^i$ with diameter $\le 1/n$ (presuming the space $X$ is such that partitions like this exist for any $n$) and denote by $\mu_\epsilon^i$ the normalized restriction of the measure $\mu_\epsilon$ to $X^i$. Then put $$ \nu_n = \sum_{i,j} \mu(X^i\times X^j) \mu_1^i\times \mu_2^j \;. $$

  • $\begingroup$ Nice and simple. I wish I had thought about this. $\endgroup$ Jan 7 '20 at 19:07
  • $\begingroup$ Thank you, Iosif $\endgroup$
    – R W
    Jan 7 '20 at 20:42
  • $\begingroup$ This is the same construction as used in the second paper I linked to in my answer. $\endgroup$
    – Steve
    Jan 7 '20 at 21:10

There are various papers where this question occurs. I guess a paper which directly covers the case you are interested in is https://arxiv.org/pdf/1901.07407.pdf . Note that here, the marginals don't have to be one dimensional.

A more general case in polish spaces is studied in https://arxiv.org/pdf/1811.00304.pdf Proposition 2. Since there is no Lebesgue measure on polish spaces, here the question is about absolute continuity with respect to the product measure of the marginals. Since in your case the product measure is absolutely continuous with respect to the Lebesgue measure, this of course also answers your problem.

There are other papers which incorporate this problem, mostly in the context of regularized optimal transport, but I do not have the references at hand.

Note also that in your case, with one dimensional marginals, things are simple as the concept of quantile functions and copulas can be used (similar in spirit to Iosif's answer) .


$\newcommand{\R}{\mathbb R}$ Here is an idea of the desired construction. You have a Borel measure $\pi$ on $\R^n$ with marginals $\mu_1,\dots,\mu_n$. Let $X:=(X_1,\dots,X_n)$ be a random point in $\R^n$ with distribution $\pi$. For each $j=1,\dots,n$, let $U_j:=F_j(X_j)$, where $F_j$ is the cumulative distribution function of $X_j$, so that $U_j$ is uniformly distributed on the interval $(0,1)$, which we will identify with the additive group $G:=\R/\mathbb Z$. So, $U:=(U_1,\dots,U_n)$ is a random element of the group $G^n$ with uniform marginals. Let $V:=(V_1,\dots,V_n)$ be an independent copy of $U$. For each natural $k$, let $W_k=(W_{k,1},\dots,W_{k,n}):=U\oplus(V/k)$, where $\oplus$ denotes the addition in the group $G^n$, so that $W_k$ is a random element of the group $G^n$ with an absolutely continuous distribution and uniform marginals.

Finally, let $\pi_k$ be the distribution of the random point $X_k:=(X_{k,1},\dots,X_{k,n})$ in $\R^n$, where $X_{k,j}:=F_j^{-1}(W_{k,j})$ and $$F_j^{-1}(u):=\inf\{x\in\R\colon F_j(x)\ge u\}=\min\{x\in\R\colon F_j(x)\ge u\} $$ for $u\in(0,1)$. Then for each $k$ the probability measure $\pi_k$ will be absolutely continuous with marginals $\mu_1,\dots,\mu_n$, and $\pi_k$ will weakly converge to $\pi$ as $k\to\infty$.


If you work on compact domains instead of $\RR$ (or if you assume that all marginals have compact support), you can use couplings from regularized optimal transport: For $n=1$ (i.e. a coupling of just two marginals) there are two popular ways.

Entropic regularization: Use $\pi_k$ defined by $$\newcommand{\RR}{\mathbb{R}} \min_\pi \int_{\RR^2} cd\pi + \gamma_k\int_{\RR^2}\pi(\log(\pi)-1)d\lambda $$ where $\lambda$ denotes the Lebesgue measure, $c$ is some cost function (e.g. continuous and non-negative), and $\gamma_k$ is a sequence of regularization parameters converging to zero and the minimum is taken over all measures $\pi$ which have a density with respect to the Lebesgue-measure (and the density has finite entropy), which have their support in the product of the supports of the marginals, and which have the marginals $\mu_1$ and $\mu_2$. If $\gamma_k$ tends to zero it should hold that $\pi_k$ converges weakly (in the sense of measures) to $\pi$.

Quadratic regularization: Use $\pi_k$ defined by $$\newcommand{\RR}{\mathbb{R}} \min_\pi \int_{\RR^2} c d\pi + \gamma_k\int_{\RR^2}\pi^2 d\lambda $$ where $\lambda$ denotes the Lebesgue measure, $c$ is again some cost function (e.g. square integrable and non-negative) and $\gamma_k$ is a sequence of regularization parameters converging to zero and the minimum is taken over all measures $\pi$ which have a square integrable density with respect to the Lebesgue-measure, having their support in the product of the supports of the marginals and which have the marginals $\mu_1$ and $\mu_2$. I guess that weak convergence is also true here.

For entropic regularization I recommend Computational Optimal Transport (free version here) and for quadratic regularization I only have my own paper Quadratically Regularized Optimal Transport (also on the arXiv).

  • $\begingroup$ You have both $\int c\, d\pi$ and $\int\pi^2 d\lambda$. Is your $\pi$ a measure or a density? $\endgroup$ Jan 7 '20 at 14:20
  • $\begingroup$ Also, in your paper you say that a unique minimizer exists "since the objective is continuous, coercive, and strongly convex". But how can you ensure that the minimizer is a probability density (noting that the constant $1$ is not in $L^2(\mathbb R^2)$)? $\endgroup$ Jan 7 '20 at 14:32
  • $\begingroup$ For the first comment: I use $\pi$ for both. For the second: Oops, we needed compact domains (at least finite with finite Lebesgue measure), so not sure if anything works here. I adapt my answer! $\endgroup$
    – Dirk
    Jan 7 '20 at 15:14
  • $\begingroup$ @Dirk It's great that you mentioned regularization. The problem I have at hand is related somewhat to entropic regularization. I was wondering if this approach will allow us to control $H(\pi_k|\nu)$? What I mean is that if $\pi$ is not absolutely continuous wrt to $\nu$ then $H(\pi_k|\nu)$ will also go to infinity (because of lower semi continuity of $H(\cdot|\nu)$) but can we say choose $\pi_k$ cleverly so that $\frac{1}{k}H(\pi_k|\nu)\to 0$? $\endgroup$ Jan 8 '20 at 8:55
  • $\begingroup$ I would like to know the answer to this, too! $\endgroup$
    – Dirk
    Jan 8 '20 at 10:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.