# Reversing the order of conditioning in a sum to compare conditional variances

Suppose $$Z=X+Y$$ where $$X$$ is independent of $$Y$$ and $$Y\sim N(0,1)$$. I would like to compare $$\text{var}(E(X|Z))$$ to $$\text{var}(E(Z|X))$$. Obviously, $$\text{var}(E(Z|X))=\text{var}(X)$$.

In particular, my guess is that $$\text{var}(E(Z|X)) > \text{var}(E(X|Z))$$. If $$Z$$ is normal, then this is easy to prove directly from known formulas for conditional distributions of a multivariate Gaussian. Furthermore, if $$Y$$ is non-normal, then this appears to be false as well. The difficulty, as I see it, is in getting a handle on $$E(X|Z)$$ while accounting for the fact that $$Y$$ is Gaussian. Another approach is to compute (or control somehow) the marginal $$P(Z)$$, which would help understand $$P(X|Z)=P(X,Z)/P(Z)$$.

• @S.Surace: It's definitely true that if $\operatorname{Var}[X \mid Z]=0$ then $X$ is $Z$-measurable, i.e. $X =f(Z)$. The last part I have to think about. – Nate Eldredge Jun 3 '20 at 23:27
• This is a nice question. The issue can be reformulated as finding a lower bound on the estimation error of $X$ given $Z$. Now, if the law of $X$ has finite fisher information, then the Gaussianity of $Y$ together with Van-Tree's version of Cramer-Rao would give a strictly positive lower bound on the variance. This however does not prove the general case. – ofer zeitouni Jun 6 '20 at 11:27
• @PeteJorgensen Yes. The question you are asking can be rephrased as a question about the mean square error of the optimal estimator of $X$ given $Z$. Namely, your conjecture is equivalent to the claim that $\sigma^2:=\inf_{W} E(X-W)^2>0$, where the infimum is over all $Z$-measurable, $L^2$ random variables (the infimum is achieved by $W=E(X|Z)$, and this is the link to your question). $\sigma^2$ is the variance I refer to in my answer. – ofer zeitouni Jun 8 '20 at 8:22
• @PeteJorgensen Now, there is a lower bound on $\sigma^2$, known as the van-Trees version of the Cramer Rao lower bound from statistics, which gives a lower bound on $\sigma^2$ in terms of the inverse of a sum of two Fisher informations: one related to $p(Z|X)$ (which involves the Gaussianity of $Y$, and hence is bounded), and the other is the Fisher information of the law of $X$. If the latter is bounded (e.g., if $X$ has a nice smooth density that does not vanish too fast at a point) then the latter is bounded and you get a lower bound on $\sigma^2. – ofer zeitouni Jun 8 '20 at 8:23 • @PeteJorgensen The argument for the general case (ie when the density of$X$does not have a bounded Fisher information) escapes me. – ofer zeitouni Jun 8 '20 at 8:24 ## 1 Answer EDIT: It seems that one can remove the integrability condition. Suppose that $$\text{Var}[X]<\infty$$ and $$Z=X+Y$$, where $$Y\sim\mathcal{N}(0,1)$$ and $$X$$ and $$Y$$ are independent. Because of the law of total variance, $$\text{Var}\{E[Z|X]\}=\text{Var}\{X\}\\=E[\text{Var}\{X|Z\}]+ \text{Var}\{E[X|Z]\}\geq \text{Var}\{E[X|Z]\},$$ with equality iff $$\text{Var}\{X|Z\}=0$$. So to prove your conjecture we only need to show $$\text{Var}\{X|Z\}>0$$. Theorem $$\text{Var}\{X|Z\}=0$$ if and only if $$X$$ is constant. Proof: If $$X$$ is constant then $$\text{Var}[X]=0$$ and therefore also $$\text{Var}\{X|Z\}=0$$. For the converse, suppose that $$\text{Var}\{X|Z\}=0$$. Let $$\mathbb{P}$$ be the measure under which $$Z=X+Y$$, where $$X$$ has law $$P_X$$, $$Y$$ is standard Gaussian and independent of $$X$$, and $$\mathbb{\tilde P}$$ be the measure under which $$Z$$ is standard Gaussian and independent of $$X$$, which has the same law as under $$\mathbb{P}$$. Let $$L := \exp\left[X Z-\frac12 X^2\right].$$ Since $$L$$ is nonnegative, we have by Tonelli's theorem and the MGF of $$\mathcal{N}(0,1)$$ that $$\mathbb{\tilde{E}}[L]=\int_{\mathbb{R}^2}e^{xz-\frac12 x^2}\mathbb{\tilde P}_X(dx)\otimes \mathbb{\tilde P}_Z(dz)=\int_{\mathbb{R}}e^{-\frac12 x^2}\left(\int_{\mathbb{R}}e^{xz}\mathbb{\tilde P}_Z(dz)\right)\mathbb{\tilde P}_X(dx)\\=\int_{\mathbb{R}}e^{-\frac12 x^2}e^{\frac12 x^2}\mathbb{\tilde P}_X(dx) = 1.$$ Thus $$\mathbb{P}\ll\mathbb{\tilde P}$$, $$\frac{d\mathbb{P}}{d\mathbb{\tilde P}} = L$$, and we have an abstract Bayes formula $$\mathbb{E}[\varphi(X)|Z]=\frac{\mathbb{\tilde E}[\varphi(X)L|Z]}{\mathbb{\tilde E}[L|Z]}.$$ Let $$M(Z):=\mathbb{\tilde E}[L|Z]$$, which is finite by assumption. Then the conditional variance of $$X$$ given $$Z$$ can be written as $$\text{Var}\{X|Z\}=\frac{M''(Z)}{M(Z)}-\left(\frac{M'(Z)}{M(Z)}\right)^2 = \frac{d}{dz}\frac{M'(z)}{M(z)} \Bigg|_{z=Z}.$$ Since $$\text{Var}\{X|Z\}=0$$, we have $$\frac{M'(Z)}{M(Z)} = \frac{M'(0)}{M(0)} = \frac{\mathbb{E}[Xe^{-\frac12 X^2}]}{\mathbb{E}[e^{-\frac12 X^2}]} .$$ But we also have $$X = \mathbb{E}\left[ X|Z\right] = \frac{M'(Z)}{M(Z)}.$$ It therefore follows that $$X$$ is constant. • I think you must use normalcy of Y somewhere because of this: if X takes only 1/4, 3/4 and Y only 0,1, then when you see X+Y you know both X and Y so E(X|Z) = X and you don't get strict inequality. – mike Jun 4 '20 at 9:54 • Yes, @mike, I noticed that this does not get us much further. At present I do not know how to complete the argument. – S.Surace Jun 4 '20 at 12:22 • @S.Surace Nice! just a quick remark: since$zX-X^2$is bounded above, its expectation is also bounded above, so the condition you wrote holds trivially. On the other hand, you need a bit more I think, namely (as a minimum) you need the expectation of$dP/d\tilde P$to be bounded, and for that you need$E(e^X^2/2)\$ finite. – ofer zeitouni Jun 8 '20 at 14:33
• @oferzeitouni, thanks! I made a correction including the Novikov-like condition. – S.Surace Jun 8 '20 at 15:40