3
$\begingroup$

Suppose $Z=X+Y$ where $X$ is independent of $Y$ and $Y\sim N(0,1)$. I would like to compare $\text{var}(E(X|Z))$ to $\text{var}(E(Z|X))$. Obviously, $\text{var}(E(Z|X))=\text{var}(X)$.

In particular, my guess is that $\text{var}(E(Z|X)) > \text{var}(E(X|Z))$. If $Z$ is normal, then this is easy to prove directly from known formulas for conditional distributions of a multivariate Gaussian. Furthermore, if $Y$ is non-normal, then this appears to be false as well. The difficulty, as I see it, is in getting a handle on $E(X|Z)$ while accounting for the fact that $Y$ is Gaussian. Another approach is to compute (or control somehow) the marginal $P(Z)$, which would help understand $P(X|Z)=P(X,Z)/P(Z)$.

$\endgroup$
7
  • 1
    $\begingroup$ @S.Surace: It's definitely true that if $\operatorname{Var}[X \mid Z]=0$ then $X$ is $Z$-measurable, i.e. $X =f(Z)$. The last part I have to think about. $\endgroup$ – Nate Eldredge Jun 3 '20 at 23:27
  • 1
    $\begingroup$ This is a nice question. The issue can be reformulated as finding a lower bound on the estimation error of $X$ given $Z$. Now, if the law of $X$ has finite fisher information, then the Gaussianity of $Y$ together with Van-Tree's version of Cramer-Rao would give a strictly positive lower bound on the variance. This however does not prove the general case. $\endgroup$ – ofer zeitouni Jun 6 '20 at 11:27
  • 1
    $\begingroup$ @PeteJorgensen Yes. The question you are asking can be rephrased as a question about the mean square error of the optimal estimator of $X$ given $Z$. Namely, your conjecture is equivalent to the claim that $\sigma^2:=\inf_{W} E(X-W)^2>0$, where the infimum is over all $Z$-measurable, $L^2$ random variables (the infimum is achieved by $W=E(X|Z)$, and this is the link to your question). $\sigma^2$ is the variance I refer to in my answer. $\endgroup$ – ofer zeitouni Jun 8 '20 at 8:22
  • 1
    $\begingroup$ @PeteJorgensen Now, there is a lower bound on $\sigma^2$, known as the van-Trees version of the Cramer Rao lower bound from statistics, which gives a lower bound on $\sigma^2$ in terms of the inverse of a sum of two Fisher informations: one related to $p(Z|X)$ (which involves the Gaussianity of $Y$, and hence is bounded), and the other is the Fisher information of the law of $X$. If the latter is bounded (e.g., if $X$ has a nice smooth density that does not vanish too fast at a point) then the latter is bounded and you get a lower bound on $\sigma^2. $\endgroup$ – ofer zeitouni Jun 8 '20 at 8:23
  • 1
    $\begingroup$ @PeteJorgensen The argument for the general case (ie when the density of $X$ does not have a bounded Fisher information) escapes me. $\endgroup$ – ofer zeitouni Jun 8 '20 at 8:24
3
$\begingroup$

EDIT: It seems that one can remove the integrability condition.

Suppose that $\text{Var}[X]<\infty$ and $Z=X+Y$, where $Y\sim\mathcal{N}(0,1)$ and $X$ and $Y$ are independent. Because of the law of total variance, $$\text{Var}\{E[Z|X]\}=\text{Var}\{X\}\\=E[\text{Var}\{X|Z\}]+ \text{Var}\{E[X|Z]\}\geq \text{Var}\{E[X|Z]\},$$ with equality iff $ \text{Var}\{X|Z\}=0$. So to prove your conjecture we only need to show $ \text{Var}\{X|Z\}>0$.

Theorem $\text{Var}\{X|Z\}=0$ if and only if $X$ is constant.

Proof: If $X$ is constant then $\text{Var}[X]=0$ and therefore also $\text{Var}\{X|Z\}=0$. For the converse, suppose that $\text{Var}\{X|Z\}=0$. Let $\mathbb{P}$ be the measure under which $Z=X+Y$, where $X$ has law $P_X$, $Y$ is standard Gaussian and independent of $X$, and $\mathbb{\tilde P}$ be the measure under which $Z$ is standard Gaussian and independent of $X$, which has the same law as under $\mathbb{P}$. Let $$ L := \exp\left[X Z-\frac12 X^2\right].$$ Since $L$ is nonnegative, we have by Tonelli's theorem and the MGF of $\mathcal{N}(0,1)$ that $$\mathbb{\tilde{E}}[L]=\int_{\mathbb{R}^2}e^{xz-\frac12 x^2}\mathbb{\tilde P}_X(dx)\otimes \mathbb{\tilde P}_Z(dz)=\int_{\mathbb{R}}e^{-\frac12 x^2}\left(\int_{\mathbb{R}}e^{xz}\mathbb{\tilde P}_Z(dz)\right)\mathbb{\tilde P}_X(dx)\\=\int_{\mathbb{R}}e^{-\frac12 x^2}e^{\frac12 x^2}\mathbb{\tilde P}_X(dx) = 1.$$

Thus $\mathbb{P}\ll\mathbb{\tilde P}$, $ \frac{d\mathbb{P}}{d\mathbb{\tilde P}} = L$, and we have an abstract Bayes formula $$\mathbb{E}[\varphi(X)|Z]=\frac{\mathbb{\tilde E}[\varphi(X)L|Z]}{\mathbb{\tilde E}[L|Z]}.$$ Let $M(Z):=\mathbb{\tilde E}[L|Z]$, which is finite by assumption. Then the conditional variance of $X$ given $Z$ can be written as $$\text{Var}\{X|Z\}=\frac{M''(Z)}{M(Z)}-\left(\frac{M'(Z)}{M(Z)}\right)^2 = \frac{d}{dz}\frac{M'(z)}{M(z)} \Bigg|_{z=Z}.$$ Since $\text{Var}\{X|Z\}=0$, we have $$\frac{M'(Z)}{M(Z)} = \frac{M'(0)}{M(0)} = \frac{\mathbb{E}[Xe^{-\frac12 X^2}]}{\mathbb{E}[e^{-\frac12 X^2}]} . $$ But we also have $$ X = \mathbb{E}\left[ X|Z\right] = \frac{M'(Z)}{M(Z)}.$$ It therefore follows that $X$ is constant.

$\endgroup$
4
  • 1
    $\begingroup$ I think you must use normalcy of Y somewhere because of this: if X takes only 1/4, 3/4 and Y only 0,1, then when you see X+Y you know both X and Y so E(X|Z) = X and you don't get strict inequality. $\endgroup$ – mike Jun 4 '20 at 9:54
  • $\begingroup$ Yes, @mike, I noticed that this does not get us much further. At present I do not know how to complete the argument. $\endgroup$ – S.Surace Jun 4 '20 at 12:22
  • $\begingroup$ @S.Surace Nice! just a quick remark: since $zX-X^2$ is bounded above, its expectation is also bounded above, so the condition you wrote holds trivially. On the other hand, you need a bit more I think, namely (as a minimum) you need the expectation of $dP/d\tilde P$ to be bounded, and for that you need $E(e^X^2/2)$ finite. $\endgroup$ – ofer zeitouni Jun 8 '20 at 14:33
  • $\begingroup$ @oferzeitouni, thanks! I made a correction including the Novikov-like condition. $\endgroup$ – S.Surace Jun 8 '20 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.