The sign of the mean curvature on convex cones in three dimensions

Question

My question is as follows:

It is known that a closed smooth curve in $\mathbb{R}^2$ is convex iff its (signed) curvature has a constant sign. I wonder if one can characterize smooth convex cones in $\mathbb{R}^3$ in a similar way.

Here is the precise statement. I say that an open set $V$ is a cone if $tV \subset V$ for any $t>0$. I say that a cone is smooth if its boundary $\partial V$ is non-empty and, say, $C^\infty$ (except at the origin). So I can define the mean curvature $H$ on the smooth part of the boundary of $V$.

Do we have the equivalence between: (1) $V$ is convex, and (2) $H$ has a constant sign on $\partial V$?

Answer 1

The second fundamental form $II$ of $\partial V$ vanishes in the radial direction (along any line that goes through the origin), so one eigenvalue of $II$ is zero. There are only two eigenvalues since $\dim(\partial V)=2$ and the mean curvature is (up to a factor of 2) the trace of $II$. Therefore the nonzero eigenvalue of $II$ has the same sign as the mean curvature.

We have thus that $II$ is positive semidefinite iff the mean curvature is positive (and similarly for negative). Positive semidefiniteness of $II$ is equivalent with convexity, so the answer to your question is yes.

Be careful with the last assertion, though. For a submanifold on a general Riemannian manifold positive semidefiniteness of $II$ is not enough to guarantee convexity in the geodesic sense. Convexity does always imply $II\geq0$.

Let's see why things work in our case. Let $\gamma:[a,b]\to\mathbb R^3\setminus\{0\}$ be any line segment, and suppose $\gamma(a),\gamma(b)\in V$. Let $\phi:\mathbb R^3\setminus\{0\}\to S^2$ be the projection to the sphere, $\phi(x)=x/|x|$. Now $\phi\circ\gamma:[a,b]\to S^2$ is an arc of a great circle on the sphere. The second fundamental form on $\partial V$ is positive semidefinite and vanishes in the radial direction, so $II$ on $\partial\phi(V)$ is also positive semidefinite. It is an easier exercise to see that this implies that $\phi(V)$ is convex (in the sense that a minimal geodesic joining any two points in the set stays in the set). Thus $\phi\circ\gamma$ only takes values in $\phi(V)$. Since $V$ is conical, this means that $\gamma$ only takes values in $V$.