About the suboptimality of linear estimators

Question

Let $X$ be a random variable and $N$ a Gaussian noise independent from $X$. We observe $Y=X+N$ and want to estimate $X$ based on $Y$ to minimize the mean square error $mmse(X|Y):=E(\hat X(Y)-X)^2$.

We know the optimal estimator is the conditional expectation $E(X|Y)$ and it is linear on $Y$ iff $X$ is also Gaussian. But it is clear that in general we have

$mmse(X|Y)\leq mmse_L(X|Y)$

here $mmse_L(X|Y)$ denotes the minimum mean square error caused by a linear estimator, i.e. $\hat X(Y)=\alpha Y$ for some $\alpha$.

Now assume we have another random variable $Z$ which may depended on $X$ but independent from the noise $N$. Then we can estimate $X$ using $Y$ and $Z$ together.

My question is, does it hold in general that

$mmse(X|Y)-mmse(X|Y,Z)\leq mmse_L(X|Y)- mmse_L(X|Y,Z)$

where $mmse(X|Y,Z)$ and $mmse_L(X|Y,Z)$ denotes the minimum mean square error of the optimal estimator using $(Y,Z)$ and the linear estimator (linear both on $Y$ and $Z$), respectively.

If we rewrite it as

$mmse_L(X|Y,Z)-mmse(X|Y,Z)\leq mmse_L(X|Y)-mmse(X|Y)$

it would mean that the suboptimality (in terms of the absolute value, may not in terms of the percentage though) of the linear estimator decreases if we have more information for the estimation.

Answer 1

Take $Z$ to be uncorrelated with $X,N$ but dependent on $X$. For example, take $X$ to be the square of a Gaussian and set $Z=X B$ where $B$ is an independent $+/- 1$ Bernoulli. Then $mmse_L(X|Y,Z)=mmse_L(X|Y)$ because the linear estimator given $Y$ or given $Y,Z$ is the same. So the right side of your claimed inequality is $0$, whereas the left side is easily shown to be strictly positive (take for example the variance of $N$ to infinity, then $mmse(X|Y)=var(X)$ while $mmse(X|Y,Z)\leq mmse(X|Z)\leq mmse(X|\, |X|)=0$).

In short, your inequality is false in general.