4
$\begingroup$

Hello, I am trying to find an upper bound on the expectation value of the product of two random variables.

So suppose x, y are two non-independent random variables, given that I know the distribution of x p(x) and the distribution of y q(y), how can I find an upper bound on E[|x * y |] that is a function of p and q?

I know that Holder's inequality gives an upper bound to my problem in terms of moments of x and y, but this is a poor bound for the problem that I am considering.

Thank you! Best Michele

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Cauchy-Schwarz? $\endgroup$
    – Yemon Choi
    Commented Oct 8, 2012 at 0:22
  • 1
    $\begingroup$ Well if it gives you a poor bound for your problem, you need to specify more details. The Cauchy-Schwarz inequality is sharp $\endgroup$
    – Yemon Choi
    Commented Oct 8, 2012 at 1:42
  • 2
    $\begingroup$ Why the down-votes? I don't think that C-S is sharp for this situation. If you assume that they are non-negative valued, the sharp upper bound is obtained when the variables are monotonically coupled. I'll post a formula for this in a few minutes. $\endgroup$
    – Anthony Quas
    Commented Oct 8, 2012 at 3:04
  • 2
    $\begingroup$ C-S is sharp if all that you know are the second moments. Here we've got far more information: the entire distribution of the random variables. $\endgroup$
    – Anthony Quas
    Commented Oct 8, 2012 at 3:07
  • 1
    $\begingroup$ I still think, though, that the question should have included at least some examples of the kinds of distribution that the OP had in mind $\endgroup$
    – Yemon Choi
    Commented Oct 8, 2012 at 3:52

2 Answers 2

Reset to default
7
$\begingroup$

I'll assume that $X$ and $Y$ are non-negative random variables. Let $F_X$ be the cumulative distribution function of $X$ (that is $F_X(t)=\mathbb P(X\le t)$) and $F_Y$ be the cumulative distribution function of $Y$.

In your notation, probably $F_X(t)=\int_0^t p(s)\,ds$ and $F_Y(y)=\int_0^t q(t)\,dt$.

Now define two functions on $[0,1]$: $g_X(x)=\sup\lbrace t\colon \mathbb P(X\le t)\le x\rbrace $ and similarly $g_Y(x)=\sup\lbrace t\colon \mathbb P(Y\le t)\le x\rbrace$. These functions are the increasing rearrangements of $X$ and $Y$. That is these are non-decreasing functions with the property that $m\lbrace x\colon g_X(x)\le t\rbrace =\mathbb P(X\le t)$ and $m\lbrace x\colon g_Y(x)\le t\rbrace = \mathbb P(Y\le t)$.

Now the largest possible value of $\mathbb E XY$ given the distributions is $\int_0^1 g_X(t)g_Y(t)\ dt$. Intuitively the reason for this is that the largest value for the expectation is obtained when the largest values of $X$ are multiplied by the largest values of $Y$. Slightly more precisely imagine you've arranged the $X$ values from largest to smallest. Think of these as "weights" for the $Y$ values. Obviously you get the biggest integral if you weight the big $Y$ values with the biggest weights.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Dear Anthony, Thank you very much for your answer! A few questions: - Is this bound better than Holder's inequality's bound 𝔼[XY] <= E[X^p]^(1/p)*E[Y^q]^(1/q) with q>1,p>1,1/p+1/q = 1? If it is, is there a way of proving or simply justifying this? - Where can I find a proof of the bound that you suggested? Thanks you Best Michele $\endgroup$
    – James
    Commented Oct 9, 2012 at 23:54
  • 1
    $\begingroup$ The justification is in my answer. For more, you could try Lindvall's book "Lectures on the Coupling Method". This is the best possible bound: If you let $\omega$ be uniformly distributed in the unit interval, then $g_X(\omega)$ has the same distribution as $X$ and $g_Y(\omega)$ has the same distribution as $Y$ and the product of these random variables has the integral in my answer. $\endgroup$
    – Anthony Quas
    Commented Oct 10, 2012 at 5:19
  • $\begingroup$ Dear Anthony, Still, it is not clear to me how to prove the inequality that you suggested : E[X*Y] <= \int_{0}^{1} dt g_{X}(t) * g_{Y}(t). Is the proof in the book "Lectures on the Coupling Method"? It it not clear either wether and why this bound is better than the Holder's inequality bound E[X^p]^(1/p)*E[Y^q]^(1/q). Can you prove this? Thanks! Michele $\endgroup$
    – James
    Commented Oct 16, 2012 at 1:05
  • $\begingroup$ If you know that $X$ is uniformly distributed on the unit interval and $Y$ are is the uniformly distributed random variable on [1,2], then the bound I'm suggesting comes from $g_X(t)=t$, $g_Y(t)=1+t$, so that $\mathbb E XY\le \int (t+t^2)\,dt=5/6$. If you use H\"older's inequality, you get $(1/(p+1))^{1/p}((2^{q+1}-1)/(q+1))^{1/q}$. This is greater than 5/6 for all $1/p+1/q=1$. My bound is attained if $X$ is uniform and $Y=1+X$. In general, my bound is always attained for some joint distribution on $X$ and $Y$. The Holder bound is not always attained. So mine is lower and is best poss. $\endgroup$
    – Anthony Quas
    Commented Oct 16, 2012 at 5:57
  • $\begingroup$ Apparently the inequality I'm quoting goes by the name "Hardy-Littlewood inequality". See math.toronto.edu/almut/rearrange.pdf $\endgroup$
    – Anthony Quas
    Commented Oct 16, 2012 at 6:55
3
$\begingroup$

I would try yo apply Hoeffding's Lemma, who used his result to identify the bivariate cdfs with given marginal cdfs that minimize or maximize correlation. Let $(X,Y)$ be a random vector with bivariate cdf $H$, let $F$ and $G$ be their marginal cdfs, respectively. It is well known that a sharp upper bound for $H(x,y)$ is $\min(F(x),G(y))$. By Hoeffding's Lemma we get that $$E(XY)\leq E(X)E(Y)+\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\big[\min(F(x),G(y))-F(x)G(y)\big]dxdy$$

Improve this answer
$\endgroup$
1

Not the answer you're looking for? Browse other questions tagged .