2
$\begingroup$

Given absolutely continuous random variables $(X, Y)$ with joint distribution $P_{XY}$, we construct $Z:=\sqrt{\gamma} Y+N_\mathsf{G}$ where $N_\mathsf{G}\sim N(0, 1)$ and is independent of $(X,Y)$. The conditional variance of $Y$ given $Z$ is defined as $$\mathsf{var}(Y|Z)=\mathbb{E}[(Y-\mathbb{E}[Y|Z])^2|Z].$$ Let $Y_x$ be a random variable having distribution $P_{Y|X=x}$ and accordingly $Z_x=\sqrt{\gamma} Y_x+N_\mathsf{G}$. I need to show the inequality $$\mathbb{E}[\mathsf{var}^2(Y|Z)]\geq \int \mathbb{E}[\mathsf{var}^2(Y_x|Z_x)]P_X(\text{d}x).$$

In other words, what I need to show is

$$\mathbb{E}[\mathsf{var}^2(Y|Z)]\geq \mathbb{E}[\mathsf{var}^2(Y|Z, X)].$$

$\endgroup$
  • $\begingroup$ Does Jensen's inequality help? $\endgroup$ – usul Feb 24 '16 at 8:52
  • $\begingroup$ It seems that Jensen's inequality might help but I cannot see how to apply it.. $\endgroup$ – math-Student Feb 24 '16 at 14:47
1
$\begingroup$

The inequality does not hold in general. Indeed, without loss of generality $\gamma=1$; otherwise, replace $\sqrt{\gamma} Y$ by $Y$, so that $Z=Y+N$, where $N:=N_\mathsf{G}$. Let us write $E$ and $V$ for $\mathbb{E}$ and $\mathsf{var}$, respectively.

First here, the heuristics. Suppose that, in an appropriate sense, $Y$ is much smaller than $N$. Then $Z$ depends mainly on $N$, so that $Z$ is almost independent of the pair $(X,Y)$, so that the pair $(X,Y)$ is almost independent of $Z$, whence $V(Y|Z)\approx V(Y)$ and $V(Y|X,Z)\approx V(Y|X)$. So, it is "almost enough" to construct $X$ and $Y$ such that $$\tag{1} V(Y)^2<E V(Y|X)^2,$$ which will hold if the conditional variance $V(Y|X)$ depends "very much" on $X$. For instance, let $X$ take only values $1$ and $0$, with probabilities $p$ and $q:=1-p$, respectively, for some $p\in(0,1)$. Let then $Y:=RX$, where $R$ is a Rademacher random variable (r.v.) independent of $X$, so that, in particular, $P(R=\pm1)=1/2$. Then $V(Y)=E X^2=EX=p$ and $V(Y|X)=X^2=X$, so that $E V(Y|X)^2=EX^2=p$, and so, $(1)$ holds.


It is straightforward but a bit tedious to make this heuristics rigorous. Let $X$, $R$, and $N$ be as before, at that making sure that these three r.v.'s are independent. Let here then $Y:=kRX$ for some real $k>0$, which will be later chosen to be small enough, in accordance with the above heuristics. Then $Z=kRX+N$. Also, $Y^2=k^2X$, and so, $$E(Y^2|X,Z)=E(Y^2|X)=k^2X.$$ Note that $$E(X|Z\in dz)=P(X=1|Z\in dz)=\frac{P(X=1,kR+N\in dz)}{P(X=1,kR+N\in dz)+P(X=0,N\in dz)} $$ $$ =\frac{P(X=1)\,P(kR+N\in dz)}{P(X=1)\,P(kR+N\in dz)+P(X=0)\,P(N\in dz)} $$ $$ =g(z):=\frac{p\,[f(z-k)+f(z+k)]}{p\,[f(z-k)+f(z+k)]+2q\,f(z)} <\frac{p\,\cosh kz}{p\,\cosh kz+2q}, $$ where $f$ is the pdf of $N(0,1)$. The above reasoning is not quite rigorous. However, now knowing the function $g$, one can check that indeed $EXh(Z)=Eg(Z)h(Z)$ for all nonnegative Borel functions $h$, or just for $h$ being (say) the indicators of the intervals of the form $(-\infty,t]$ for real $t$.

It follows that
$$0\le E(X|Z)<\frac p{2q}\,\cosh kZ.$$ Since $Y^2=k^2X$, it follows that $$\tag{2} E[V(Y|Z)^2]\le E[E(Y^2|Z)^2] <k^4(\tfrac p{2q})^2\,E\cosh^2 kZ<k^4(\tfrac pq)^2 $$ by dominated convergence if $k$ is small enough.

The calculation of $E(Y|X,Z)$ is similar to the above one of $E(X|Z)$, resulting in the formula $$E(Y|X,Z)=kX\frac{f(Z-k)-f(Z+k)}{f(Z-k)+f(Z+k)}=kX\tanh kZ. $$ Recalling again that $E(Y^2|X,Z)=k^2X$ and $X^2=X$, we have $V(Y|X,Z)=k^2X(1-\tanh^2 kZ)$, whence, by dominated convergence, $$k^{-4}E[V(Y|X,Z)^2]=E[X(1-\tanh^2 kZ)^2]\to EX=p $$ as $k\downarrow0$.

Comparing this with $(2)$, we see that for small enough $p\in(0,1)$ and $k>0$ $$E[V(Y|X,Z)^2]>E[V(Y|Z)^2]. $$

It only remains to note that, if you insist that $(X, Y)$ be absolutely continuous, more straightforward and tedious approximations will be needed.

$\endgroup$
  • $\begingroup$ Iosif, thanks a lot for your nice and detailed answer. This question was raised in an attempt to show that the map $\gamma\mapsto \mathbb{E}[V(Y|Z_{\gamma}, X)]-\mathbb{E}[V(Y|Z_{\gamma})]$ is increasing. Could you please let me know what you think about this? When $\gamma=0$ then $\mathbb{E}[V(Y|Z_{0}, X)]-\mathbb{E}[V(Y|Z_{0})]\leq 0$ and when $\gamma$ is sufficiently large, then $\mathbb{E}[V(Y|Z_{\gamma}, X)]-\mathbb{E}[V(Y|Z_{\gamma})]$ is very close to zero. So it is intuitive (to me) that this map is indeed increasing. $\endgroup$ – math-Student Feb 25 '16 at 2:32
  • $\begingroup$ Sorry, I am going away from my desk for about two weeks. $\endgroup$ – Iosif Pinelis Feb 25 '16 at 3:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.