# An Inequality Regarding the Squared Conditional Variance

Given absolutely continuous random variables $(X, Y)$ with joint distribution $P_{XY}$, we construct $Z:=\sqrt{\gamma} Y+N_\mathsf{G}$ where $N_\mathsf{G}\sim N(0, 1)$ and is independent of $(X,Y)$. The conditional variance of $Y$ given $Z$ is defined as $$\mathsf{var}(Y|Z)=\mathbb{E}[(Y-\mathbb{E}[Y|Z])^2|Z].$$ Let $Y_x$ be a random variable having distribution $P_{Y|X=x}$ and accordingly $Z_x=\sqrt{\gamma} Y_x+N_\mathsf{G}$. I need to show the inequality $$\mathbb{E}[\mathsf{var}^2(Y|Z)]\geq \int \mathbb{E}[\mathsf{var}^2(Y_x|Z_x)]P_X(\text{d}x).$$

In other words, what I need to show is

$$\mathbb{E}[\mathsf{var}^2(Y|Z)]\geq \mathbb{E}[\mathsf{var}^2(Y|Z, X)].$$

• Does Jensen's inequality help? – usul Feb 24 '16 at 8:52
• It seems that Jensen's inequality might help but I cannot see how to apply it.. – math-Student Feb 24 '16 at 14:47

The inequality does not hold in general. Indeed, without loss of generality $\gamma=1$; otherwise, replace $\sqrt{\gamma} Y$ by $Y$, so that $Z=Y+N$, where $N:=N_\mathsf{G}$. Let us write $E$ and $V$ for $\mathbb{E}$ and $\mathsf{var}$, respectively.

First here, the heuristics. Suppose that, in an appropriate sense, $Y$ is much smaller than $N$. Then $Z$ depends mainly on $N$, so that $Z$ is almost independent of the pair $(X,Y)$, so that the pair $(X,Y)$ is almost independent of $Z$, whence $V(Y|Z)\approx V(Y)$ and $V(Y|X,Z)\approx V(Y|X)$. So, it is "almost enough" to construct $X$ and $Y$ such that $$\tag{1} V(Y)^2<E V(Y|X)^2,$$ which will hold if the conditional variance $V(Y|X)$ depends "very much" on $X$. For instance, let $X$ take only values $1$ and $0$, with probabilities $p$ and $q:=1-p$, respectively, for some $p\in(0,1)$. Let then $Y:=RX$, where $R$ is a Rademacher random variable (r.v.) independent of $X$, so that, in particular, $P(R=\pm1)=1/2$. Then $V(Y)=E X^2=EX=p$ and $V(Y|X)=X^2=X$, so that $E V(Y|X)^2=EX^2=p$, and so, $(1)$ holds.

It is straightforward but a bit tedious to make this heuristics rigorous. Let $X$, $R$, and $N$ be as before, at that making sure that these three r.v.'s are independent. Let here then $Y:=kRX$ for some real $k>0$, which will be later chosen to be small enough, in accordance with the above heuristics. Then $Z=kRX+N$. Also, $Y^2=k^2X$, and so, $$E(Y^2|X,Z)=E(Y^2|X)=k^2X.$$ Note that $$E(X|Z\in dz)=P(X=1|Z\in dz)=\frac{P(X=1,kR+N\in dz)}{P(X=1,kR+N\in dz)+P(X=0,N\in dz)}$$ $$=\frac{P(X=1)\,P(kR+N\in dz)}{P(X=1)\,P(kR+N\in dz)+P(X=0)\,P(N\in dz)}$$ $$=g(z):=\frac{p\,[f(z-k)+f(z+k)]}{p\,[f(z-k)+f(z+k)]+2q\,f(z)} <\frac{p\,\cosh kz}{p\,\cosh kz+2q},$$ where $f$ is the pdf of $N(0,1)$. The above reasoning is not quite rigorous. However, now knowing the function $g$, one can check that indeed $EXh(Z)=Eg(Z)h(Z)$ for all nonnegative Borel functions $h$, or just for $h$ being (say) the indicators of the intervals of the form $(-\infty,t]$ for real $t$.

It follows that
$$0\le E(X|Z)<\frac p{2q}\,\cosh kZ.$$ Since $Y^2=k^2X$, it follows that $$\tag{2} E[V(Y|Z)^2]\le E[E(Y^2|Z)^2] <k^4(\tfrac p{2q})^2\,E\cosh^2 kZ<k^4(\tfrac pq)^2$$ by dominated convergence if $k$ is small enough.

The calculation of $E(Y|X,Z)$ is similar to the above one of $E(X|Z)$, resulting in the formula $$E(Y|X,Z)=kX\frac{f(Z-k)-f(Z+k)}{f(Z-k)+f(Z+k)}=kX\tanh kZ.$$ Recalling again that $E(Y^2|X,Z)=k^2X$ and $X^2=X$, we have $V(Y|X,Z)=k^2X(1-\tanh^2 kZ)$, whence, by dominated convergence, $$k^{-4}E[V(Y|X,Z)^2]=E[X(1-\tanh^2 kZ)^2]\to EX=p$$ as $k\downarrow0$.

Comparing this with $(2)$, we see that for small enough $p\in(0,1)$ and $k>0$ $$E[V(Y|X,Z)^2]>E[V(Y|Z)^2].$$

It only remains to note that, if you insist that $(X, Y)$ be absolutely continuous, more straightforward and tedious approximations will be needed.

• Iosif, thanks a lot for your nice and detailed answer. This question was raised in an attempt to show that the map $\gamma\mapsto \mathbb{E}[V(Y|Z_{\gamma}, X)]-\mathbb{E}[V(Y|Z_{\gamma})]$ is increasing. Could you please let me know what you think about this? When $\gamma=0$ then $\mathbb{E}[V(Y|Z_{0}, X)]-\mathbb{E}[V(Y|Z_{0})]\leq 0$ and when $\gamma$ is sufficiently large, then $\mathbb{E}[V(Y|Z_{\gamma}, X)]-\mathbb{E}[V(Y|Z_{\gamma})]$ is very close to zero. So it is intuitive (to me) that this map is indeed increasing. – math-Student Feb 25 '16 at 2:32
• Sorry, I am going away from my desk for about two weeks. – Iosif Pinelis Feb 25 '16 at 3:40