Return to Answer

$\newcommand\ep\epsilon$Getting rid of the instances of $\approx$, one can state the question as follows:

Let $f(X):=E(Y|X)$, where $X$ and $Y$ are random variables (r.v.'s) ($X$ possibly a multivariate one) such that $EY^2<\infty$. Note that $E\ep=0$ and $Cov(f(X),\ep)=0$, where $\ep:=Y-f(X)$.

Suppose that for a sequence $(g_n)$ of Borel-measurable functions one has $g_n(X)\to f(X)$ in probability (as $n\to\infty$). Does it then follow that \begin{equation*} Cov(g_n(X),\ep_n)\to0, \tag{1}\label{1} \end{equation*} where $\ep_n:=Y-g_n(X)$?

The answer to this question is: Of course, not.

Indeed, let e.g. $X$ be a r.v. uniformly distributed on the interval $[-1,1]$, and let $Y:=X$, so that $f(X)=X$ and $\ep=0$.

Let \begin{equation*} g_n(X):=X\,1(|X|\ge1/n)+n^2 X\,1(|X|<1/n). \end{equation*} Then $g_n(X)\to X=f(X)$ in probability.

However, $Eg_n(X)=0$, $\ep_n=Y-g_n(X)=X-g_n(X)=(1-n^2)X\,1(|X|<1/n)$, and hence \begin{equation*} Cov(g_n(X),\ep_n)=Eg_n(X)\ep_n=n^2(1-n^2)\,EX^2\,1(|X|<1/n) \\ =(1-n^2)/(3n)\to-\infty\ne0. \end{equation*} So, \eqref{1} fails to hold. $\quad\Box$

On the other hand, if $g_n(X)\to f(X)$ in $L^2$, then it is easy to see that \eqref{1} will hold.

Details on this: We have $\|g_n(X)-f(X)\|_2\to0$, where $\|Z\|_2:=\sqrt{EZ^2}$. We also have the identity \begin{equation*} g_n(X)\ep_n=g_n(X)(Y-g_n(X))=f(X)(Y-f(X))+Y(g_n(X)-f(X))+2f(X)(f(X)-g_n(X))-(g_n(X)-f(X))^2. \end{equation*} Taking here the expectations and recalling that $Ef(X)(Y-f(X))=Ef(X)\ep=Cov(f(X),\ep)=0$, we get \begin{equation*} Eg_n(X)\ep_n=EY(g_n(X)-f(X))+2Ef(X)(f(X)-g_n(X))-E(g_n(X)-f(X))^2. \tag{2}\label{2} \end{equation*} By the Cauchy--Schwarz inequality and the condition $\|g_n(X)-f(X)\|_2\to0$, \begin{equation*} |EY(g_n(X)-f(X))|\le\|Y\|_2\,\|g_n(X)-f(X)\|_2\to0. \tag{3}\label{3} \end{equation*} Note all that \begin{equation} \|f(X)\|_2\le\|Y\|_2<\infty \tag{3.5}\label{3.5} \end{equation} since $f(X)$ is an orthogonal projection of $Y$ in $L^2$. So, \begin{equation*} |Ef(X)(f(X)-g_n(X))|\le\|f(X)\|_2\,\|g_n(X)-f(X)\|_2\to0. \tag{4}\label{4} \end{equation*} Also, \begin{equation*} E(g_n(X)-f(X))^2=\|g_n(X)-f(X)\|_2^2\to0. \tag{5}\label{5} \end{equation*} Collecting \eqref{2}--\eqref{5}, we get \begin{equation*} Eg_n(X)\ep_n\to0. \tag{6}\label{6} \end{equation*} Next, $\ep_n=Y-g_n(X)=Y-f(X)+f(X)-g_n(X)=\ep+f(X)-g_n(X)$. Taking here the expectations and recalling that $E\ep=0$, we get \begin{equation} |E\ep_n|=|E(f(X)-g_n(X))|\le\|g_n(X)-f(X)\|_2\to0. \tag{7}\label{7} \end{equation} Further, $|Eg_n(X)-Ef(X)|\le E|g_n(X)-Ef(X)|\le\|g_n(X)-Ef(X)\|_2\to0$, again by the Cauchy--Schwarz inequality and the condition $\|g_n(X)-f(X)\|_2\to0$. So, \begin{equation} Eg_n(X)\to Ef(X), \tag{8}\label{8} \end{equation} and $|Ef(X)|\le\|f(X)\|_2<\infty$ by the Cauchy--Schwarz inequality and \eqref{3.5}. By \eqref{6}--\eqref{8}, \begin{equation} Cov(g_n(X),\ep_n)=Eg_n(X)\ep_n-Eg_n(X)\,E\ep_n\to0-Ef(X)\times 0=0, \end{equation} so that \eqref{1} holds. $\quad\Box$

$\newcommand\ep\epsilon\newcommand{\de}{\delta}$Getting rid of the instances of $\approx$, one can state the question as follows:

Let $f(X):=E(Y|X)$, where $X$ and $Y$ are random variables (r.v.'s) ($X$ possibly a multivariate one) such that $EY^2<\infty$. Note that $E\ep=0$ and $Cov(f(X),\ep)=0$, where $\ep:=Y-f(X)$.

Suppose that for a sequence $(g_n)$ of Borel-measurable functions one has $g_n(X)\to f(X)$ in probability (as $n\to\infty$). Does it then follow that \begin{equation*} Cov(g_n(X),\ep_n)\to0, \tag{1}\label{1} \end{equation*} where $\ep_n:=Y-g_n(X)$?

The answer to this question is: Of course, not.

Indeed, let e.g. $X$ be a r.v. uniformly distributed on the interval $[-1,1]$, and let $Y:=X$, so that $f(X)=X$ and $\ep=0$.

Let \begin{equation*} g_n(X):=X\,1(|X|\ge1/n)+n^2 X\,1(|X|<1/n). \end{equation*} Then $g_n(X)\to X=f(X)$ in probability.

However, $Eg_n(X)=0$, $\ep_n=Y-g_n(X)=X-g_n(X)=(1-n^2)X\,1(|X|<1/n)$, and hence \begin{equation*} Cov(g_n(X),\ep_n)=Eg_n(X)\ep_n=n^2(1-n^2)\,EX^2\,1(|X|<1/n) \\ =(1-n^2)/(3n)\to-\infty\ne0. \end{equation*} So, \eqref{1} fails to hold. $\quad\Box$

On the other hand, if $g_n(X)\to f(X)$ in $L^2$, then it is easy to see that \eqref{1} will hold.

Details on this: We have $\|g_n(X)-f(X)\|_2\to0$, where $\|Z\|_2:=\sqrt{EZ^2}$. We also have the identity \begin{equation*} g_n(X)\ep_n=g_n(X)(Y-g_n(X))=f(X)(Y-f(X))+Y(g_n(X)-f(X))+2f(X)(f(X)-g_n(X))-(g_n(X)-f(X))^2. \end{equation*} Taking here the expectations and recalling that $Ef(X)(Y-f(X))=Ef(X)\ep=Cov(f(X),\ep)=0$, we get \begin{equation*} Eg_n(X)\ep_n=EY(g_n(X)-f(X))+2Ef(X)(f(X)-g_n(X))-E(g_n(X)-f(X))^2. \tag{2}\label{2} \end{equation*} By the Cauchy--Schwarz inequality and the condition $\|g_n(X)-f(X)\|_2\to0$, \begin{equation*} |EY(g_n(X)-f(X))|\le\|Y\|_2\,\|g_n(X)-f(X)\|_2\to0. \tag{3}\label{3} \end{equation*} Note all that \begin{equation} \|f(X)\|_2\le\|Y\|_2<\infty \tag{3.5}\label{3.5} \end{equation} since $f(X)$ is an orthogonal projection of $Y$ in $L^2$. So, \begin{equation*} |Ef(X)(f(X)-g_n(X))|\le\|f(X)\|_2\,\|g_n(X)-f(X)\|_2\to0. \tag{4}\label{4} \end{equation*} Also, \begin{equation*} E(g_n(X)-f(X))^2=\|g_n(X)-f(X)\|_2^2\to0. \tag{5}\label{5} \end{equation*} Collecting \eqref{2}--\eqref{5}, we get \begin{equation*} Eg_n(X)\ep_n\to0. \tag{6}\label{6} \end{equation*} Next, $\ep_n=Y-g_n(X)=Y-f(X)+f(X)-g_n(X)=\ep+f(X)-g_n(X)$. Taking here the expectations and recalling that $E\ep=0$, we get \begin{equation} |E\ep_n|=|E(f(X)-g_n(X))|\le\|g_n(X)-f(X)\|_2\to0. \tag{7}\label{7} \end{equation} Further, $|Eg_n(X)-Ef(X)|\le E|g_n(X)-Ef(X)|\le\|g_n(X)-Ef(X)\|_2\to0$, again by the Cauchy--Schwarz inequality and the condition $\|g_n(X)-f(X)\|_2\to0$. So, \begin{equation} Eg_n(X)\to Ef(X), \tag{8}\label{8} \end{equation} and $|Ef(X)|\le\|f(X)\|_2<\infty$ by the Cauchy--Schwarz inequality and \eqref{3.5}. By \eqref{6}--\eqref{8}, \begin{equation} Cov(g_n(X),\ep_n)=Eg_n(X)\ep_n-Eg_n(X)\,E\ep_n\to0-Ef(X)\times 0=0, \end{equation}

In view of \eqref{3}, \eqref{3.5}, \eqref{4}, \eqref{5}, and \eqref{7}, one can also get an explicit bound on $|Cov(g_n(X),\ep_n)|$: \begin{equation} |Cov(g_n(X),\ep_n)|\le3y\de_n+2\de_n^2, \end{equation} where $y:=\|Y\|_2$ and $\de_n:=\|g_n(X)-f(X)\|_2$.

$\newcommand\ep\epsilon$Getting rid of the instances of $\approx$, one can state the question as follows:

Let $f(X):=E(Y|X)$, where $X$ and $Y$ are random variables (r.v.'s) ($X$ possibly a multivariate one) such that $EY^2<\infty$. Note that $E\ep=0$ and $Cov(f(X),\ep)=0$, where $\ep:=Y-f(X)$.

Suppose that for a sequence $(g_n)$ of Borel-measurable functions one has $g_n(X)\to f(X)$ in probability (as $n\to\infty$). Does it then follow that \begin{equation*} Cov(g_n(X),\ep_n)\to0, \tag{1}\label{1} \end{equation*} where $\ep_n:=Y-g_n(X)$?

The answer to this question is: Of course, not.

Indeed, let e.g. $X$ be a r.v. uniformly distributed on the interval $[-1,1]$, and let $Y:=X$, so that $f(X)=X$ and $\ep=0$.

Let \begin{equation*} g_n(X):=X\,1(|X|\ge1/n)+n^2 X\,1(|X|<1/n). \end{equation*} Then $g_n(X)\to X=f(X)$ in probability.

However, $Eg_n(X)=0$, $\ep_n=Y-g_n(X)=X-g_n(X)=(1-n^2)X\,1(|X|<1/n)$, and hence \begin{equation*} Cov(g_n(X),\ep_n)=Eg_n(X)\ep_n=n^2(1-n^2)\,EX^2\,1(|X|<1/n) \\ =(1-n^2)/(3n)\to-\infty\ne0. \end{equation*} So, \eqref{1} fails to hold. $\quad\Box$

On the other hand, if $g_n(X)\to f(X)$ in $L^2$, then it is easy to see that \eqref{1} will hold.

$\newcommand\ep\epsilon$Getting rid of the instances of $\approx$, one can state the question as follows:

Let $f(X):=E(Y|X)$, where $X$ and $Y$ are random variables (r.v.'s) ($X$ possibly a multivariate one) such that $EY^2<\infty$. Note that $E\ep=0$ and $Cov(f(X),\ep)=0$, where $\ep:=Y-f(X)$.

Suppose that for a sequence $(g_n)$ of Borel-measurable functions one has $g_n(X)\to f(X)$ in probability (as $n\to\infty$). Does it then follow that \begin{equation*} Cov(g_n(X),\ep_n)\to0, \tag{1}\label{1} \end{equation*} where $\ep_n:=Y-g_n(X)$?

The answer to this question is: Of course, not.

Indeed, let e.g. $X$ be a r.v. uniformly distributed on the interval $[-1,1]$, and let $Y:=X$, so that $f(X)=X$ and $\ep=0$.

Let \begin{equation*} g_n(X):=X\,1(|X|\ge1/n)+n^2 X\,1(|X|<1/n). \end{equation*} Then $g_n(X)\to X=f(X)$ in probability.

However, $Eg_n(X)=0$, $\ep_n=Y-g_n(X)=X-g_n(X)=(1-n^2)X\,1(|X|<1/n)$, and hence \begin{equation*} Cov(g_n(X),\ep_n)=Eg_n(X)\ep_n=n^2(1-n^2)\,EX^2\,1(|X|<1/n) \\ =(1-n^2)/(3n)\to-\infty\ne0. \end{equation*} So, \eqref{1} fails to hold. $\quad\Box$

On the other hand, if $g_n(X)\to f(X)$ in $L^2$, then it is easy to see that \eqref{1} will hold.

Details on this: We have $\|g_n(X)-f(X)\|_2\to0$, where $\|Z\|_2:=\sqrt{EZ^2}$. We also have the identity \begin{equation*} g_n(X)\ep_n=g_n(X)(Y-g_n(X))=f(X)(Y-f(X))+Y(g_n(X)-f(X))+2f(X)(f(X)-g_n(X))-(g_n(X)-f(X))^2. \end{equation*} Taking here the expectations and recalling that $Ef(X)(Y-f(X))=Ef(X)\ep=Cov(f(X),\ep)=0$, we get \begin{equation*} Eg_n(X)\ep_n=EY(g_n(X)-f(X))+2Ef(X)(f(X)-g_n(X))-E(g_n(X)-f(X))^2. \tag{2}\label{2} \end{equation*} By the Cauchy--Schwarz inequality and the condition $\|g_n(X)-f(X)\|_2\to0$, \begin{equation*} |EY(g_n(X)-f(X))|\le\|Y\|_2\,\|g_n(X)-f(X)\|_2\to0. \tag{3}\label{3} \end{equation*} Note all that \begin{equation} \|f(X)\|_2\le\|Y\|_2<\infty \tag{3.5}\label{3.5} \end{equation} since $f(X)$ is an orthogonal projection of $Y$ in $L^2$. So, \begin{equation*} |Ef(X)(f(X)-g_n(X))|\le\|f(X)\|_2\,\|g_n(X)-f(X)\|_2\to0. \tag{4}\label{4} \end{equation*} Also, \begin{equation*} E(g_n(X)-f(X))^2=\|g_n(X)-f(X)\|_2^2\to0. \tag{5}\label{5} \end{equation*} Collecting \eqref{2}--\eqref{5}, we get \begin{equation*} Eg_n(X)\ep_n\to0. \tag{6}\label{6} \end{equation*} Next, $\ep_n=Y-g_n(X)=Y-f(X)+f(X)-g_n(X)=\ep+f(X)-g_n(X)$. Taking here the expectations and recalling that $E\ep=0$, we get \begin{equation} |E\ep_n|=|E(f(X)-g_n(X))|\le\|g_n(X)-f(X)\|_2\to0. \tag{7}\label{7} \end{equation} Further, $|Eg_n(X)-Ef(X)|\le E|g_n(X)-Ef(X)|\le\|g_n(X)-Ef(X)\|_2\to0$, again by the Cauchy--Schwarz inequality and the condition $\|g_n(X)-f(X)\|_2\to0$. So, \begin{equation} Eg_n(X)\to Ef(X), \tag{8}\label{8} \end{equation} and $|Ef(X)|\le\|f(X)\|_2<\infty$ by the Cauchy--Schwarz inequality and \eqref{3.5}. By \eqref{6}--\eqref{8}, \begin{equation} Cov(g_n(X),\ep_n)=Eg_n(X)\ep_n-Eg_n(X)\,E\ep_n\to0-Ef(X)\times 0=0, \end{equation} so that \eqref{1} holds. $\quad\Box$

$\newcommand\ep\epsilon$Getting rid of the instances of $\approx$, one can state the question as follows:

Let $f(X):=E(Y|X)$, where $X$ and $Y$ are random variables (r.v.'s) ($X$ possibly a multivariate one) such that $E|Y|<\infty$. Note that $E\ep=0$ and $Cov(f(X),\ep)=0$, where $\ep:=Y-f(X)$.

Suppose that for a sequence $(g_n)$ of Borel-measurable functions one has $g_n(X)\to f(X)$ in probability (as $n\to\infty$). Does it then follow that \begin{equation*} Cov(g_n(X),\ep_n)\to0, \tag{1}\label{1} \end{equation*} where $\ep_n:=Y-g_n(X)$?

The answer to this question is: Of course, not.

Indeed, let e.g. $X$ be a r.v. uniformly distributed on the interval $[-1,1]$, and let $Y:=X$, so that $f(X)=X$ and $\ep=0$.

Let \begin{equation*} g_n(X):=X\,1(|X|\ge1/n)+n^2 X\,1(|X|<1/n). \end{equation*} Then $g_n(X)\to X=f(X)$ in probability.

However, $Eg_n(X)=0$, $\ep_n=Y-g_n(X)=X-g_n(X)=(1-n^2)X\,1(|X|<1/n)$, and hence \begin{equation*} Cov(g_n(X),\ep_n)=Eg_n(X)\ep_n=n^2(1-n^2)\,EX^2\,1(|X|<1/n) \\ =(1-n^2)/(3n)\to-\infty\ne0. \end{equation*} So, \eqref{1} fails to hold. $\quad\Box$

$\newcommand\ep\epsilon$Getting rid of the instances of $\approx$, one can state the question as follows:

Let $f(X):=E(Y|X)$, where $X$ and $Y$ are random variables (r.v.'s) ($X$ possibly a multivariate one) such that $EY^2<\infty$. Note that $E\ep=0$ and $Cov(f(X),\ep)=0$, where $\ep:=Y-f(X)$.

Suppose that for a sequence $(g_n)$ of Borel-measurable functions one has $g_n(X)\to f(X)$ in probability (as $n\to\infty$). Does it then follow that \begin{equation*} Cov(g_n(X),\ep_n)\to0, \tag{1}\label{1} \end{equation*} where $\ep_n:=Y-g_n(X)$?

The answer to this question is: Of course, not.

Indeed, let e.g. $X$ be a r.v. uniformly distributed on the interval $[-1,1]$, and let $Y:=X$, so that $f(X)=X$ and $\ep=0$.

Let \begin{equation*} g_n(X):=X\,1(|X|\ge1/n)+n^2 X\,1(|X|<1/n). \end{equation*} Then $g_n(X)\to X=f(X)$ in probability.

However, $Eg_n(X)=0$, $\ep_n=Y-g_n(X)=X-g_n(X)=(1-n^2)X\,1(|X|<1/n)$, and hence \begin{equation*} Cov(g_n(X),\ep_n)=Eg_n(X)\ep_n=n^2(1-n^2)\,EX^2\,1(|X|<1/n) \\ =(1-n^2)/(3n)\to-\infty\ne0. \end{equation*} So, \eqref{1} fails to hold. $\quad\Box$

On the other hand, if $g_n(X)\to f(X)$ in $L^2$, then it is easy to see that \eqref{1} will hold.