# Statistical model vs. Statistical Learning Theory

I am interested in the relation between a statistical model $$(\Omega, \mathcal{F}, (\mathbb{P}^\theta : \theta\in\Theta))$$, where the hypotheses are "$$\mathbb{P}^\theta$$ is a good approximation of the real probability density" and statistical learning theory, where given input data $$X:\Omega \rightarrow \mathbb{R}^m$$ and labels $$Y:\Omega \rightarrow \mathbb{R}^n$$ we search for a function $$f$$ minimizing the risk $$\mathcal{E}(f)=\int_\Omega (f(X(\omega)) - Y(\omega))^2 ~\text{d}\mathbb{P}(\omega),$$ leading to hypotheses "$$f$$ is a good approximation of the relation between X and Y"

Are the two approaches equivalent? Can I reformulate the statistical learning approach to a statistical model, thus obtaining a set of probability measures $$\mathbb{P}_f$$ on some probability space, or are there fundamental differences?

I haven't found a resource where they rigorously actually do this. Any help or references are well appreciated.

## 1 Answer

To be able to reformulate what you call the statistical learning approach, you need to make some assumptions on the joint law $$(X,Y)$$, and how it is related to the function $$f$$. For instance, you may assume that the following relation holds (regression model): $$Y = f(X) + \varepsilon,$$ where $$f$$ belongs to some family $$F$$ of functions (parameterized or not), and $$\varepsilon$$ is independent from $$X$$, with some assumptions on its law (typically that it is centered, with a prescribed variance). Then, calling $$\mathbb{P}_f$$ the law of the couple $$(X,Y)$$, and assuming some measurability, the minimizer $$\hat{f}$$ of $$\mathcal{E}(f)$$ on $$F$$ is an estimator of $$f$$ in the statistical model $$(\Omega, \mathcal{F},(\mathbb{P}_f,\ f\in F))$$. Note that depending on the assumptions made on the class $$F$$, this estimator might or might not have good statistical properties.

If you want more information, you should probably check for the keyword "nonparametric regression".