Consider two closed convex cones $A$ and $B$ in $\mathbb{R}^3$. Assume that they are convex even without zero vector, i.e. $A \setminus \{0\}$ and $B \setminus \{0\}$ are also convex (it helps to avoid weird cases like a plane being convex cone). Suppose that they do not have common directions, i.e. $A \cap B = \{0\}$. It seems that in this case there must exist a strict separating plane, i.e. there exists some unit vector $d$ such that $d \cdot a > 0$ for $a \in A \setminus \{0\}$ and $d \cdot b < 0$ for $b \in B \setminus \{0\}$.

I have tried to prove it myself, but to no avail. If we take a unit sphere $S$, then we may consider convex sets $\mathcal{H}(A \cap S)$ and $\mathcal{H}(B \cap S)$. They are compact and disjoint, so perpendicular bisector between the pair of closest points should separate these sets. If these two closest points are both on unit sphere $S$, then the bisector passes through origin and everything is OK. But in other cases the bisector is not helpful.

After some searching, I have found several topics which may be related to the question:

  1. Hahn–Banach separation theorem gives separating plane for convex sets. However, the plane provided may touch both cones.
  2. Farkas's lemma also provides a separating plane for cones. But in classical formulation one cone must be a ray, and the other one must be cone hull of finite number of vectors. Also, the separating plane constructed is nonstrict for one of the cones.
  3. $A \cap S$ and $B \cap S$ should be geodesically convex on the unit sphere $S$. Since they do not intersect, there might be a strict separating big circle between them. I'm not even sure how to treat this sort of objects though.

I would be glad to see either a reference to literature or a simple proof.


I think you may be misreading the Hahn-Banach theorem in this case, as it should give you a strict separating plane here. Anyway, you can avoid this problem by enlarging the cones a bit before applying Hahn-Banach. This is no problem in the finite dimensional case.

Here's what I believe is a direct proof: Find a codimension-1 hyperplane $H$ such that the cross-section $A\cap H$ is compact and convex (such and $H$ should exist by your hypotheses). The intersection $B \cap H$ will be closed and convex, though perhaps not compact. Treat $H$ itself as a linear space and apply Hahn-Banach to strictly separate $A\cap H$ from $B\cap H$ by a codimension-1 (in $H$) hyperplane $G$. Complete $G$ to a codimension-1 (in the ambient space) hyperplane passing through the origin, which will separate the cones $A\setminus \{0\}$ and $B\setminus \{0\}$ as you wanted.

In the infinite dimensional case, it is of course more difficult and requires stronger hypotheses (one of the cones has to be closed and the other have a compact cross-section) and there is some literature on this. For example:

Separation Properties of Convex Cones, by V. L. Klee, Jr. PAMS 6 313-318 (1955)

  • $\begingroup$ Your reference fits perfectly! Obviously, $A' = A \cap -A = \{0\}$ for any cone I have. Hence Theorem 2.7 proves exactly what I need, even in a more general case. $\endgroup$ – stgatilov Oct 9 '15 at 14:28
  • $\begingroup$ I'm not sure your direct proof works. Even if you can cut properly the cone $A$ with a plane $H$, this cutting plane may not even intersect the $B$ cone, so $G$ would separate a convex set from nothing. Take two opposite $45^{\circ}$ circular cones as $A$ and $B$, for example. $\endgroup$ – stgatilov Oct 9 '15 at 14:32
  • $\begingroup$ As for Hahn-Banach theorem, I cannot see how to apply it directly (at least with wikipedia formulations). I think it is not possible, because some additional knowledge about cones is necessary to prove strict separation: simply having closed convex cones is not enough (recall a plane being a cone), it is necessary to know that opposite vectors are not present in each of the cones, I guess. $\endgroup$ – stgatilov Oct 9 '15 at 14:37
  • $\begingroup$ The idea with enlarging the cones is great, I really missed it! The cones can be enlarged by small angle. It is easy to show that enlarged cones are disjoint, also it is easy to find a separating plane. However, the main difficulty remains in proving that a cone can be enlarged without losing convexity. And this question is not so clear, I'm afraid. $\endgroup$ – stgatilov Oct 9 '15 at 14:41
  • 1
    $\begingroup$ You need the existence of a well-behaved cross-section for the individual cones as a hypothesis, no way around this. Another way to phrase this is to ask the cone not to contain any affine subspace of positive dimension (a salient cone). Non-salient cones non-trivially intersect every codim-1 subspace, so cannot be separated from anything. $\endgroup$ – Igor Khavkine Oct 10 '15 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.