0
$\begingroup$

I am considering the following model: $(X_i,Y_i)_{i=1}^n$ are iid random pairs with $(X_i,Y_i)\in[0,1]^2$. Let $f(x)=\mathbb{E}[Y|X=x]$. Consider an estimate $\hat{f}_n$ of $f$.

Under some hypothesis on $f$ (such as Holder), I am looking for a result providing an upper bound (as good as possible) on $\Vert\hat{f}_n-f\Vert_\infty$ with high probability.

I am particularly interested in the case of the regressogram:

Let $I_1,\dots,I_K$ be the regular partition of the unit interval. Suppose $x\in I_k$, then $$\hat{f}_n(x) = \frac{\sum_{i=1}^nY_i1(X_i\in I_k)}{\sum_{i=1}^n1(X_i\in I_k)}.$$

Is there any reference?

Improve this question
$\endgroup$
6
  • $\begingroup$ If you also assume $Y$ has bounded range (say, in $[0,1]$) then you can use covering numbers to give finite-sample bounds, as done, say, here: ieeexplore.ieee.org/document/7944658 $\endgroup$
    – Aryeh Kontorovich
    Commented Apr 19, 2020 at 20:07
  • $\begingroup$ @AryehKontorovich Thanks for you answer! Is there anything about the regressogram? I am surprised I cannot find anything in the literature (it is probably the simplest regression estimator), maybe I don't have the right keywords. $\endgroup$
    – ess
    Commented Apr 19, 2020 at 20:59
  • $\begingroup$ @AryehKontorovich PS: I am not sure I understand what the paper does. I guess the result I am looking for is given by theorem 10 or corrolary 11. But the results are on $R_n(h,q)$, how do they transfer to $h$ (which is the equivalent of my $f$)? $\endgroup$
    – ess
    Commented Apr 19, 2020 at 21:07
  • $\begingroup$ You can probably argue that the regressogram produces a Lipschitz (or maybe even Holder) smooth function, and then use covering numbers to derive generalization results. Our paper focuses on the computational aspects, in general doubling spaces, so it's probably an overkill for your needs. $\endgroup$
    – Aryeh Kontorovich
    Commented Apr 19, 2020 at 22:34
  • $\begingroup$ @AryehKontorovich Any reference to the simpler setting? $\endgroup$
    – ess
    Commented Apr 20, 2020 at 7:58

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .