# Negative population variable importance

I just state the probabilistic problem here. It is about the optimality of the conditional expectation.

Consider a random vector $(X_1,X_2,X_3,Y)$ with $Y\in L^1$, and let $f$ be the Borelian function such that $E[Y\mid X_1, X_2, X_3]=f(X_1, X_2, X_3)$.

Define $$I = E\left[{\bigl(Y-f(X'_1,X_2,X_3)\bigr)}^2\right] - E\left[{\bigl(Y-f(X_1,X_2,X_3)\bigr)}^2\right]$$ where $X'_1$ is a random variable having the same distribution as $X_1$ but is independent of all other random variables $X_2,X_3,Y$.

The question: "is it possible that $I < 0$?"

In case you are interested in a simple example, the paper Correlation and variable importance in random forests (Gregorutti & al) investigates some particular cases such as the additive case $f(X_1,X_2,X_3)=f_1(X_1)+f_2(X_2)+f_3(X_3)$. It is not difficult to get that $I=2\textrm{Var}\bigl(f_1(X_1)\bigr)$ in this case.

That is not possible: $I$ cannot be negative.
Indeed, let $X:=X_1$, $X':=X'_1$, $V:=(X_2,X_3)$. Let $E_Z$ denote the conditional expectation given a random variable/random vector $Z$. Then $E_{(X',V)}Y=E_V Y$, since $X'$ is independent of $(Y,V)$ (see details below). So, $$E(Y-f(X',V))^2\ge E(Y-E_{(X',V)}Y)^2=E(Y-E_V Y)^2 \ge E(Y-E_{(X,V)} Y)^2=E(Y-f(X,V))^2.$$ The two inequalities above hold because $E(Y-g(Z))^2\ge E(Y-E_Z Y)^2$ for any Borel function $g$.
Details on the equality $E_{(X',V)}Y=E_V Y$: Let $h(V):=E_V Y$. Let $I\{\cdot\}$ denote the indicator function. Let $A$ and $B$ denote Borel sets in appropriate spaces. Then, by the independence of $X'$ from $(Y,V)$, $$EYI\{X'\in A\}I\{V\in B\}=EI\{X'\in A\}\,EYI\{V\in B\}$$ $$=EI\{X'\in A\}\,Eh(V)I\{V\in B\} =Eh(V)I\{X'\in A\}I\{V\in B\},$$ whence $E_{(X',V)}Y=h(V)=E_V Y$.