Lower bound for reduced variance after conditioning

Question

Let $X$ be a random variable with variance $\tau^2$ and $Y$ be another random variable such that $Y-X$ is independent of $X$ and has mean zero and variance $\sigma^2$. (One can think of $Y$ as a noisy observation of $X$.) It follows from the law of total variance that $\mathbb{E}[\operatorname{Var}(X|Y)]\leq\operatorname{Var}(X)$. Under normality, it is known that $\operatorname{Var}(X|Y)=\frac{\sigma^2\tau^2}{\sigma^2+\tau^2}=\tau^2\left(1-\frac{\tau^2}{\sigma^2+\tau^2}\right)$ almost surely, and this inequality is stronger than $\mathbb{E}[\operatorname{Var}(X|Y)]\leq\operatorname{Var}(X)$ as it quantifies how much the expected variance is reduced.

I wonder if one could prove something similar in general. If this does not hold in general, would it help to assume the random variables are sub-Gaussian?

Answer 1

The LHS of the expression you wrote is the minimum mean square error (i.e., the expectation $\inf E(X-\hat X)^2$ where $\hat X$ is measurable on $Y$). On the other hand, the expression you wrote ($\hat \sigma^2:=\sigma^2 \tau^2/(\sigma^2+\tau^2)$) is the error of the optimal linear estimator, so it always bounds from above the optimal error, that is $\hat \sigma^2\geq E Var(X|Y)$; equality is achieved in the Gaussian case.