Finding Decision Boundary from empirical distribution - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-23T03:56:47Z http://mathoverflow.net/feeds/question/109378 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/109378/finding-decision-boundary-from-empirical-distribution Finding Decision Boundary from empirical distribution Ankit 2012-10-11T14:04:34Z 2013-05-10T04:22:00Z <p>Based on measuring a certain characteristic, we want to classify measurements as coming from either of two populations. The true population distributions are unknown (and we don't want to take any strong assumptions on this), but we have a number of measurements from each population providing us with two empirical distributions. Thus, we are looking for a method to determine a good (i.e., approximating the optimal) decision boundary based on the empirical distributions. Do any of you happen to know a general and accurate way to determine optimal decision boundaries given two arbitrary empirical distributions ?</p> http://mathoverflow.net/questions/109378/finding-decision-boundary-from-empirical-distribution/109381#109381 Answer by Arthur B for Finding Decision Boundary from empirical distribution Arthur B 2012-10-11T14:47:16Z 2012-10-11T14:55:00Z <p>There's always going to be a prior. Even a method that doesn't explicitly use one will be hiding one. The fact of the matter is, you expect your distributions to be simple in some respect, somewhat smooth, maybe belonging to a parametric family, etc. The more data you have, the less this assumption will matter, but it always will. You also need a prior on whether you think observations tend to come from $A$ or from $B$.</p> <p>What you will prefer hearing:</p> <p>Pick a gaussian kernel with variance $s$ and compare</p> <p>$$f_a(x) = \sum_{\mathbf{x_i} \in A} g(\mathbf{x}-\mathbf{x_i})$$ $$f_b(x) = \sum_{\mathbf{x_i} \in B} g(\mathbf{x}-\mathbf{x_i})$$</p> <p>if $f_a &lt; f_b$, assign to bin $A$, otherwise assign to bin $B$</p> <p>(note, this assumes that the cardinal of $A$ and $B$ are indicative of the prior probability of an element to be drawn from either)</p> <p>Adjust $s$ according to the variance of your empirical distributions.</p>