Finding Decision Boundary from empirical distribution

2012-10-11T14:04:34Z

Based on measuring a certain characteristic, we want to classify measurements as coming from either of two populations. The true population distributions are unknown (and we don't want to take any strong assumptions on this), but we have a number of measurements from each population providing us with two empirical distributions. Thus, we are looking for a method to determine a good (i.e., approximating the optimal) decision boundary based on the empirical distributions. Do any of you happen to know a general and accurate way to determine optimal decision boundaries given two arbitrary empirical distributions ?

Answer by Arthur B for Finding Decision Boundary from empirical distribution

2012-10-11T14:47:16Z

There's always going to be a prior. Even a method that doesn't explicitly use one will be hiding one. The fact of the matter is, you expect your distributions to be simple in some respect, somewhat smooth, maybe belonging to a parametric family, etc. The more data you have, the less this assumption will matter, but it always will. You also need a prior on whether you think observations tend to come from $A$ or from $B$.

What you will prefer hearing:

Pick a gaussian kernel with variance $s$ and compare

$$f_a(x) = \sum_{\mathbf{x_i} \in A} g(\mathbf{x}-\mathbf{x_i})$$ $$f_b(x) = \sum_{\mathbf{x_i} \in B} g(\mathbf{x}-\mathbf{x_i})$$

if $f_a < f_b$, assign to bin $A$, otherwise assign to bin $B$

(note, this assumes that the cardinal of $A$ and $B$ are indicative of the prior probability of an element to be drawn from either)

Adjust $s$ according to the variance of your empirical distributions.

Finding Decision Boundary from empirical distribution - MathOverflow

Finding Decision Boundary from empirical distribution

Answer by Arthur B for Finding Decision Boundary from empirical distribution