Statistical model vs. statistical learning theory

Question

I am interested in the relation between a statistical model $(\Omega, \mathcal{F}, (\mathbb{P}^\theta : \theta\in\Theta))$, where the hypotheses are "$\mathbb{P}^\theta$ is a good approximation of the real probability density" and statistical learning theory, where given input data $X:\Omega \rightarrow \mathbb{R}^m$ and labels $Y:\Omega \rightarrow \mathbb{R}^n$ we search for a function $f$ minimizing the risk $$\mathcal{E}(f)=\int_\Omega (f(X(\omega)) - Y(\omega))^2 ~\text{d}\mathbb{P}(\omega),$$ leading to hypotheses "$f$ is a good approximation of the relation between X and Y"

Are the two approaches equivalent? Can I reformulate the statistical learning approach to a statistical model, thus obtaining a set of probability measures $\mathbb{P}_f$ on some probability space, or are there fundamental differences?

I haven't found a resource where they rigorously actually do this. Any help or references are well appreciated.

Answer 1

To be able to reformulate what you call the statistical learning approach, you need to make some assumptions on the joint law $(X,Y)$, and how it is related to the function $f$. For instance, you may assume that the following relation holds (regression model): $$ Y = f(X) + \varepsilon,$$ where $f$ belongs to some family $F$ of functions (parameterized or not), and $\varepsilon$ is independent from $X$, with some assumptions on its law (typically that it is centered, with a prescribed variance). Then, calling $\mathbb{P}_f$ the law of the couple $(X,Y)$, and assuming some measurability, the minimizer $\hat{f}$ of $\mathcal{E}(f)$ on $F$ is an estimator of $f$ in the statistical model $(\Omega, \mathcal{F},(\mathbb{P}_f,\ f\in F))$. Note that depending on the assumptions made on the class $F$, this estimator might or might not have good statistical properties.

If you want more information, you should probably check for the keyword "nonparametric regression".