Find most densely located K points among N (N>K) points in two dimension Suppose I have N points in two dimensional space. 
I want to know which K of them are located most densely (so that area occupied by them will be least or sum of squares within cluster is least).  Area occupied is area of polygon which connects least number points in the group so that all the points in the group are either inside the polygon or are on the boundary of polygon. How can I find them?
I know K-means-algorithm but it will yield groups that will have number of points different than K. 
 A: Under the natural interpretation that the area of the convex hull of $k$ points is to be minimized,
the
question was addressed in David Eppstein's 1992 paper, "New algorithms for minimum area $k$-gons,"
In Proceedings 3rd ACM-SIAM Symposium on Discrete algorithms (SODA '92), 83-88. 

We also solve the related problem of finding a set of $k$ points with minimum area convex hull
  [among $n$ points in the plane], in time $O(n^2 \log n + k^3 n^2)$.

I think the definition using "connects least number points" is in fact the convex hull.
A: First of all, your problem is ill-posed, since you have to define the 'area occupied by the points' more precisely: Is it the area of their convex hull? Is it the area of the smallest enclosing circle? These are two possible definitions of area, but there may be more. 
On the other hand, the problem has all the flavour of an NP-hard problem, though I cannot confirm that yet. If it were so, then basically the only guaranteed algorithm would be exhaustive search in the space of all subsets of $K$ points.  
