We work over $\mathbb{R}^N$. Let $\mathbf{P}_1$ denote the hyperplane constructed using $N$ points, each of which is on a different axis (there are $N$ axes). We denote by $\mathbf{P}_2$ the convex hull of a set of $M$ points, but we don't know which of these points are the vertices. I am trying to find the minimum distance between $\mathbf{P}_1$, which can also be seen as a convex hull, and $\mathbf{P}_2$.

My approach is to define an optimization problem having an acceptable computational complexity. My question is how can we define such a problem ?

here. CGAL formulates it "as an optimization problem with linear constraints and a convex quadratic objective function." They then use an exact solver for quadratic programs. $\endgroup$ – Joseph O'Rourke Jul 18 '15 at 2:09here(section 5.1) if $P$ (or $Q$) is the set of vertices or not. In my case, for $\mathbf{P}_2$ I don't have the set of vertices alone, but a set of $M$ points which contains the vertices and other points ( which will be inside the convex hull constructed using these vertices) $\endgroup$ – tam Jul 18 '15 at 9:17