# Conditional expectation as square-loss minimizer over continuous functions

It is well-known that the conditional expectation of a square-integrable random variable $$Y$$ given another (real) random variable $$X$$ can be obtained by minimizing the mean square loss between $$Y$$ and $$f(X)$$ for all Borel-measurable functions $$f : \mathbb{R} \rightarrow \mathbb{R}$$, that is:

$$\tag{1}\mathbb{E}[Y \,|\,X] \, = \, \operatorname{argmin}_{f\in\mathcal{B}}\mathbb{E}|Y - f(X)|^2$$

for $$\mathcal{B}:=\{g : \mathbb{R} \rightarrow \mathbb{R} \mid g \ \text{ Borel measurable}\}$$.

Question: Are you aware of conditions on $$(X,Y)$$ for which in fact

$$\tag{2}\mathbb{E}[Y \,|\, X] \, = \, \operatorname{argmin}_{f\in\mathcal{C}}\mathbb{E}|Y - f(X)|^2 \quad \text{ with } \quad \mathcal{C}:= \{g : \mathbb{R}\rightarrow\mathbb{R} \mid g \ \text{ continuous} \} \ ?$$

$$\newcommand\R{\mathbb R}\newcommand\B{\mathscr B}$$The question can be restated as follows: When does there exist a continuous function $$g\colon\R\to\R$$ such that $$E(Y|X)=g(X)$$ almost surely (a.s.)?

A sufficient condition for this is as follows. Let $$\B(\R)$$ denote the Borel $$\sigma$$-algebra over $$\R$$. Let $$\R\times\B(\R)\ni(x,B)\mapsto\nu_x(B)\in\R$$ be a regular conditional distribution of $$Y$$ given $$X$$ (which exists), so that

• the function $$\R\ni x\mapsto\nu_x(B)\in\R$$ is Borel-measurable for each $$B\in\B(\R)$$,
• the function $$\B(\R)\ni B\mapsto\nu_x(B)\in\R$$ is a probability measure for each $$x\in\R$$,
• $$P(X\in A,Y\in B)=\int_A P(X\in dx)\nu_x(B)$$ for each $$(A,B)\in\B(\R)\times\B(\R)$$.

Suppose also that

• the function $$\R\ni x\mapsto\nu_x$$ is continuous wrt the topology of weak convergence of probability measures and
• for each $$x\in\R$$ there is some real $$r>0$$ such that the identity function $$\R\ni y\mapsto y\in\R$$ is uniformly integrable wrt to the set $$\{\nu_z\colon|z-x| of measures.

Then the function $$\R\ni x\mapsto g(x):=\int_\R y\nu_x(dy)$$ is continuous and $$E(Y|X)=g(X)$$ a.s.

A special case of the above sufficient condition is as follows. Suppose that the (joint) distribution of $$(X,Y)$$ is absolutely continuous wrt the Lebesgue measure on $$\B(\R^2)$$ with a joint pdf $$f_{X,Y}$$. Let $$f_X$$ be the pdf of $$X$$. Suppose that there is a nonnegative Borel-measurable function $$f_{Y|X}\colon\R^2\to\R$$ such that

• $$f_{Y|X}(x,y)$$ is continuous in $$x\in\R$$ for each $$y\in\R$$,
• $$f_{X,Y}(x,y)=f_{Y|X}(x,y)f_X(x)$$ for all $$(x,y)\in\R^2$$, and
• for each $$x\in\R$$ there is some real $$r>0$$ such that the identity function $$\R\ni y\mapsto y\in\R$$ is uniformly integrable wrt to the set $$\{\nu_z\colon|z-x| of measures, where $$\nu_z(dy):=f_{Y|X}(x,y)\,dy$$.

Then the function $$\R\ni x\mapsto g(x):=\int_\R y f_{Y|X}(x,y)\,dy$$ is continuous and $$E(Y|X)=g(X)$$ a.s.