Return to Revisions

The (more) correct definition of the $\widehat{a_i}$'s should be $$\widehat{a_i}:=\frac1n\,\sum_{j=1}^n g_i(X_j).$$

So, $$\widehat{a_i}=\int_{\mathbb R}\mu_n(t) g_i(t)\,dt,$$ where $$\mu_n(t):=\frac1n\,\sum_{j=1}^n \delta_{X_j}(t)$$ and $\delta_x$ is the Dirac probability measure at $x$, viewed as the (say) Schwartz distribution. So, the $\widehat{a_i}$'s may be viewed as the "coordinates" of the Schwartz distribution $\mu_n$ in the basis $(g_i)$ of $L_2(\mathbb R)$. So, if we had $\sum_i \widehat{a_i}^2<\infty$, we would also have $\mu_n\in L_2(\mathbb R)$, which is of course false. So, $\sum_i \widehat{a_i}^2=\infty$ for any basis $(g_i)$ of $L_2(\mathbb R)$, and hence $\sum_i (a_i-\widehat{a_i})^2=\infty$ for any real $a_i$'s such that $\sum_i a_i^2<\infty$.

Another view at why your idea of estimation of the density $f$ did not succeed is that the sequence $(\mu_n)$ of the (empirical) probability measures converges to the probability measure (say $\mu$) with density $f$ only weakly, and the probability measure $\mu_n$ does not even have a density, to converge to $f$ in any sense or to do anything else.

Generally, it appears unnatural to estimate a pdf in an $L^2$ framework. The natural framework should be $L^1$. This is the view advocated (I think persuasively) by some authors, including Devroye and Gyorfi, who write that their book "develops, from first principles, the ``natural'' theory for density estimation, L1, and shows why the classical L2 theory masks some fundamental properties of density estimates".