Suppose I have a function $f \in \mathcal C^{\alpha, L}([0,1])$, where $\mathcal C^{\alpha, L}([0,1])$ is the space of $\alpha$-smooth Hölder functions with norm $L$. I am interested in efficiently approximating $f$ using a kernel-based method; we might have \begin{align*} \widetilde f(x) = \sum_{j = 1}^k \mu_j \psi\left( \frac{x - c_j}{\sigma} \right), \end{align*} where $\psi(x)$ is something like a Gaussian. I am particularly interested in a logistic kernel $\psi(x) = e^{-x}(1 + e^{-x})^{-2}$.

I am reading a statistics paper which makes the following claim

Approximation theory tells us that for a compactly supported kernel and compactly supported $\beta$-Hölder function, being not necessarily nonnegative, the approximation error will be of order $k^{-\beta}$, provided $\sigma \sim k^{-1}$ and the weights are carefully chosen. This remains the case if both the kernel and the function to be approximated have exponential tails, as we consider in this work.

The paper itself is dealing with a density estimation problem, and concerns only power exponential kernels $\psi(x) \propto e^{-|x|^p}$.

I'm in the process of going through a textbook treatment referenced in the paper as a source for this claim (Constructive Approximation by DeVore and Lorentz), but I am wondering if anyone can provide a simple reference which will give me essentially the fact cited above, or even just with application to my logistic kernel. Also, in case it's not clear, I know basically nothing about approximation theory. Extensions of this result to $\mathcal{C}^{\alpha, L}([0,1]^d)$ using a tensor-product kernel $\psi^{(d)}(x) = \prod_{i = 1}^d \psi(x_i)$ would also be very helpful.


1 Answer 1


If you are just concerned with logistic kernels, and you are willing to put a mild assumption on the $\beta$-Hölder function $f$ to be estimated, then [Rousseau] assumes mild condition $\boldsymbol{A}_0$ and proved the $k^{-\beta}$ decay in Theorem 3.1, which is also cited by [Kruijer&Rousseau]. The good thing is that with this additional assumption the resulting approximation is explicitly given there. The downside is that a more general result in [DeVore&Lorentz] 2.6~2.7 holds.

[DeVore&Lorentz] discussed this thoroughly around Chap 2. As for tensor kernel, a popular easy-catch is [Gu]'s books on splines, which basically eliminated all theoretic complications but still point to useful results in statistics.

BTW, sometimes you need to swallow down a bit math, statistics is no more than a variant of mathematics:-)


[Kruijer&Rousseau]Kruijer, Willem, Judith Rousseau, and Aad Van Der Vaart. "Adaptive Bayesian density estimation with location-scale mixtures." Electronic Journal of Statistics 4 (2010): 1225-1257.

[Rousseau]Rousseau, Judith. "Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametric estimation of the density." The Annals of Statistics 38.1 (2010): 146-180. https://projecteuclid.org/download/pdfview_1/euclid.aos/1262271612

[Gu]Gu, Chong. Smoothing spline ANOVA models. Vol. 297. Springer Science & Business Media, 2013.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.