MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Hi, I am doing some Kernel density estimation, with a weighted points set (ie., each sample has a weight which is not necessary one), in N dimensions. Also, these samples are just in a metric space (ie., we can define a distance between them) but nothing else. For example, we cannot determine the mean of the sample points, nor the standard deviation. The Kernel is just affected by this distance, and the weight of each sample: f(x) = 1./(sum_weights) * sum(weight_i/h * Kernel(distance(x,x_i)/h))

In this context, I am trying to find a robust estimation for the kernel bandwidth 'h', possibly spatially varying, and preferably which gives an exact reconstruction on the training dataset x_i. If necessary, we could assume that the function is relatively smooth.

I tried using the distance to the first or second nearest neighbor but it gives quite bad results. I tried with leave-one-out optimization, but I have difficulties finding a good measure to optimize for in this context in N-d, so it finds very bad estimates, especially for the training samples themselves. I cannot use the greedy estimate based on the normal assumption since I cannot compute the standard deviation. I found references using covariance matrices to get anisotropic kernels, but again, it wouldn't hold in this space...

Someone has an idea or a reference ?

Thank you very much in advance!

share|cite|improve this question
A better place for your question would be – robin girard Aug 20 '10 at 6:23
thanks, I didn't know this site. I'm trying now. :) – WhitAngl Aug 20 '10 at 18:35
About using neighbors--although this seems like a reasonable idea, there is very little work in this direction. Luc Devroye has a few papers on this topic , and you'll see that just looking at the closest is inadequate. In general, cross-validation is the standard technique. If you are interested in L1 density estimates, there is an excellent book by Devroye and Lugosi titled "Combinatorial Methods in Density Estimation". – Matus Telgarsky Aug 22 '10 at 13:26

Robin is right, you can use this site. Let me ask a question, you estimate the density by the kernel estimator (Nadaraya-Watson), your writing of the estimator is not enough correct, when I copy your source and compile it with LaTeX writing is not correct, any ways the kernel estimator of the density of a functional regressor is $$ \widehat{f}(x)= \frac{\displaystyle\sum_{i=1}^{n}K\Big(h_K^{-1}d(x,X_i)\Big)Y_i}{\displaystyle\sum_{i=1}^{n}K\Big(h_K^{-1}d(x,X_i)\Big)} . $$ where $d (.,.)$ is the metric . You can refer to Ferraty et al of Sabatier University, I think you know people there (I saw your CV). So we must choose the metric $d$, to estimate the smoothing parameter $h$, I think they found a way to choose, by cross-validation, they are obtained the optimal $h$. So before starting, I would advise you to see the work of the team probability for Laboratory Toulouse. For the robustness you can see the book of Huber (1981): Robust Statistics, it is very good, or Hambel et al (1986). Good lunk.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.