2
$\begingroup$

If $X$ and $Y$ are random variables, then a maximal coupling of $X$ and $Y$ is a coupling $\left(X', Y'\right)$ such that $\mathbf{P}\left(X'=Y'\right)$ is maximal (that is, the probability that the coupled variables coincide is optimal). By $``$coupling$"$, I'm referring to a random vector $\left(X',Y'\right)$ such that $X'$ has the same distribution as $X$, and $Y'$ has the same distribution as $Y$.

Are there any benefits of using a maximal coupling to describe distributional results regarding the marginals $X$ and $Y$? E.g., are the measures of dependence between $X$ and $Y$, such as covariance, mutual information, joint entropy, Kullback-Leibler distance, or $$\lvert\mathbf{P}\left(X'\in A\text{ and }Y'\in B\right)-\mathbf{P}\left(X'\in A\right)\mathbf{P}\left(Y'\in B\right)\rvert$$ going to be extreme in the maximal coupling?

$\endgroup$
0

1 Answer 1

2
$\begingroup$

$\newcommand{\R}{\mathbb R}\newcommand{\la}{\lambda}$Let $\mu$ and $\nu$ stand for the distributions of real-valued random variables (r.v.'s) $X$ and $Y$, respectively. Let $\Pi(\mu,\nu)$ denote the set of all probability distributions over $\R^2$ with marginals $\mu$ and $\nu$.

By Strassen's Theorem 11, $\pi\in\Pi(\mu,\nu)$ is the distribution of a maximal coupling iff \begin{equation} \pi(D)=1-\|\mu-\nu\|, \end{equation} where $D:=\{(x,x)\colon x\in\R\}$ and $\|\mu-\nu\|$ is the total variation distance between $\mu$ and $\nu$.

It should be clear now that a maximal coupling is not unique. Indeed, let $(P,N)$ be the Hahn decomposition of the signed measure $\mu-\nu$ and then let $(\mu-\nu)_+$ and $(\mu-\nu)_-=(\nu-\mu)_+$ be the Jordan positive and negative parts of $\mu-\nu$, so that $(\mu-\nu)_+(A)=(\mu-\nu)(A\cap P)$ and $(\mu-\nu)_-(A)=-(\mu-\nu)(A\cap N)$ for Borel $A\subseteq\R$. Let $\mu\wedge\nu:=\mu-(\mu-\nu)_+=\nu-(\nu-\mu)_+$. Let $\la$ be any measure on $\R^2$ with marginals $(\mu-\nu)_+$ and $(\nu-\mu)_+$. For all Borel $B\subseteq\R^2$, let \begin{equation} \pi(B)=(\mu\wedge\nu)(\text{pr}(B\cap D))+\la(B), \end{equation} where $\text{pr}(x,y):=x$ for $(x,y)\in\R^2$. Then $\pi\in\Pi(\mu,\nu)$ and $\pi$ is the distribution of a maximal coupling. In particular, if $\mu$ and $\nu$ are mutually singular, then any $\pi\in\Pi(\mu,\nu)$ whatsoever is the distribution of a maximal coupling.

It should also be clear now that a maximal coupling is just a thing in itself, unlike the things you mentioned: the covariance, the mutual information, the joint entropy, the Kullback--Leibler distance.

The Kullback--Leibler distance actually depends only on the marginals $\mu$ and $\nu$; any choice of $\pi\in\Pi(\mu,\nu)$ plays no role.

The covariance depends on Euclidean distances, which are of no consequence for a maximal coupling -- which latter depends on the Hamming distance $\rho(x,y):=1(x\ne y)$. The mutual information and the joint entropy also depend on the Hamming distance, and yet in general they attain their extremes not at a maximal coupling.

E.g., suppose that $\mu(\{x\})=P(X=x)=\dfrac x{m(2m+1)}\,1(x\in[2m])$ and $\nu(\{y\})=P(Y=y)=\dfrac {2m+1-y}{m(2m+1)}\,1(y\in[2m])$, where $m$ is a positive integer and $[n]:=\{1,\dots,n\}$ -- so that $(\mu\wedge\nu)(\{x\})=\dfrac {\min(x,2m+1-x)}{m(2m+1)}\,1(x\in[2m])$. So, for a maximal coupling $(X',Y')$ we have $$P(X'=Y')=(\mu\wedge\nu)(\R)=\dfrac{m+1}{2m+1}>\dfrac12.$$ On the hand, here a coupling $(X'',Y'')$ that maximizes the covariance and the mutual information (and minimizes the joint entropy) is given by the condition $Y''=2m+1-X''$ almost surely, so that $$P(X''=Y'')=0.\quad\Box$$ We see that the extremizers of the covariance, the mutual information, and the joint entropy are very different from maximal couplings.

$\endgroup$
5
  • $\begingroup$ Thank you. What if I am seeking a maximal coupling subject to some given constraint - could it be unique in that case? $\endgroup$ Commented Aug 7, 2023 at 2:06
  • 1
    $\begingroup$ @TheSubstitute : I guess that depends on the constraint -- which I think must be very rigid to get the uniqueness. (I think so because the Hamming distance is very non-smooth and very insensitive to points, except for their being distinct.) But that is a different question, isn't it? $\endgroup$ Commented Aug 7, 2023 at 2:13
  • $\begingroup$ @TheSubstitute : Can we first wrap this up, according to the guidelines? $\endgroup$ Commented Aug 7, 2023 at 13:06
  • $\begingroup$ Can you clarify "It should also be clear now that a maximal coupling is just a thing in itself, unlike the things you mentioned"? $\endgroup$ Commented Aug 8, 2023 at 0:15
  • $\begingroup$ @TheSubstitute : This thesis is detailed in what follows it. Is there anything in those details that seems unclear? $\endgroup$ Commented Aug 8, 2023 at 1:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .