MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Based on measuring a certain characteristic, we want to classify measurements as coming from either of two populations. The true population distributions are unknown (and we don't want to take any strong assumptions on this), but we have a number of measurements from each population providing us with two empirical distributions. Thus, we are looking for a method to determine a good (i.e., approximating the optimal) decision boundary based on the empirical distributions. Do any of you happen to know a general and accurate way to determine optimal decision boundaries given two arbitrary empirical distributions ?

share|cite|improve this question

There's always going to be a prior. Even a method that doesn't explicitly use one will be hiding one. The fact of the matter is, you expect your distributions to be simple in some respect, somewhat smooth, maybe belonging to a parametric family, etc. The more data you have, the less this assumption will matter, but it always will. You also need a prior on whether you think observations tend to come from $A$ or from $B$.

What you will prefer hearing:

Pick a gaussian kernel with variance $s$ and compare

$$f_a(x) = \sum_{\mathbf{x_i} \in A} g(\mathbf{x}-\mathbf{x_i})$$ $$f_b(x) = \sum_{\mathbf{x_i} \in B} g(\mathbf{x}-\mathbf{x_i})$$

if $f_a < f_b$, assign to bin $A$, otherwise assign to bin $B$

(note, this assumes that the cardinal of $A$ and $B$ are indicative of the prior probability of an element to be drawn from either)

Adjust $s$ according to the variance of your empirical distributions.

share|cite|improve this answer

There are many well known ways to do this: logistic regression, probit regression, Bayesian logistic regression, Bayesian probit regression, Fisher's linear discriminant, perceptron, backpropigation networks, Bayesian neural networks, SVM, RVM, Bayesian graphical networks, Markov random fields...

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.