0
$\begingroup$

Let $X$ be a random variable with variance $\tau^2$ and $Y$ be another random variable such that $Y-X$ is independent of $X$ and has mean zero and variance $\sigma^2$. (One can think of $Y$ as a noisy observation of $X$.) It follows from the law of total variance that $\mathbb{E}[\operatorname{Var}(X|Y)]\leq\operatorname{Var}(X)$. Under normality, it is known that $\operatorname{Var}(X|Y)=\frac{\sigma^2\tau^2}{\sigma^2+\tau^2}=\tau^2\left(1-\frac{\tau^2}{\sigma^2+\tau^2}\right)$ almost surely, and this inequality is stronger than $\mathbb{E}[\operatorname{Var}(X|Y)]\leq\operatorname{Var}(X)$ as it quantifies how much the expected variance is reduced.

I wonder if one could prove something similar in general. If this does not hold in general, would it help to assume the random variables are sub-Gaussian?

$\endgroup$
1
$\begingroup$

The LHS of the expression you wrote is the minimum mean square error (i.e., the expectation $\inf E(X-\hat X)^2$ where $\hat X$ is measurable on $Y$). On the other hand, the expression you wrote ($\hat \sigma^2:=\sigma^2 \tau^2/(\sigma^2+\tau^2)$) is the error of the optimal linear estimator, so it always bounds from above the optimal error, that is $\hat \sigma^2\geq E Var(X|Y)$; equality is achieved in the Gaussian case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.