# Submanifolds lying on the boundary of a convex domain

Let $M$ be a submanifold of $\mathbb R^n$. Call $M$ locally convex if locally $M$ is contained in the boundary of a convex domain of $\mathbb R^n$.

Is there any known condition that is equivalent to local convexity?

Some special cases are easy to treat. If the second fundamental form with respect to some normal direction is strictly positive definite, then $M$ is locally convex. Vice versa, if the second fundamental forms with respect to all normal directions are indefinite, $M$ is not locally convex.

Also the case of hypersurfaces is solved: $M$ is locally convex if and only if the second fundamental form, with respect to a smooth nowhere vanishing normal vector field, is positive semidefinite at every point.

-
You need to formulate your question more carefully... You say "M is locally convex if and only if the second fundamental form is semidefinite at every point"; what about graph $y=x^3$ in $\mathbb R^2$. – Anton Petrunin Feb 15 '11 at 14:24
Thanks, corrected. – user175348 Feb 15 '11 at 14:49

First another example (elaborating Petrunin's comment) of a curve in $\mathbb R^3$ that is not locally convex: consider $x \mapsto (x, x^3, x^5)$. From the first derivative, any support plane at the origin needs to contain the $x$-axis. From the third derivative, if there's a supporot plane it needs to be the $xy$-plane. However, from the fifth derivative, the curve crosses this plane. Therefore, it is not locally convex. On the other hand, the curves $(x, x^3, x^4)$ and $(x, x^9, x^{132})$ are locally convex at the origin.

This example shows what you need to do: express the submanifold as the graph of a function from the tangent bundle to the normal bundle. Look at successive $k$-jets of the function. Typically you can determine local convexity from the 2-jet, that is, the second fundamental form. If not, you can look at the higher and higher jets, that is, polynomial approximations of greater and greater degree. If there is a linear functional on the normal bundle that when composed with this polynomial is strictly positive in a neighborhood of the origin, then it's locally convex at that point. If the submanifold is only $C^\infty$ with infinite order contact to some subspace of the tangent space, you're out of luck with this approach---even if you dutifully spend an infinite amount of time computing polynomial approximations, you will never resolve it. These polynomials, of course, can be expressed in terms of the 2nd fundamental form and its covariant derivatives.

Depending on how the submanifold is defined near $x$, the question of local convexity might be an algorithmically unsolvable question (depending on assumptions about the form of the input data).

-
Thanks for the answer. I still have a question though. Even assuming that we are in a "nice" case (e.g. $M$ is real-analytic) how do i know how many jet spaces i should consider? If for example $M$ contains a straight line, for any order of approximation and any linear functional strict positivity will fail. – user175348 Feb 16 '11 at 19:18
@Andrea Altomani: Good point. I think it depends exactly how you're given the information. If you're told that the function is analytic, it's not enough, as Brouwer-style examples show: e.g. in the plane take the graph of the function $\Sum(a_i x^{2i})$ where $a_i$ = 1 if the i'th element of an enumeration of all possible formal deductions in some formal system yields the Riemann hypothesis, $a_i = -1$ if the $i$th zero gives a counterexample, and 0 otherwise. It's analytic, but there's no way to know when to stop checking. But more ypically you'd have some other way to understand a curve – Bill Thurston Feb 17 '11 at 5:53

$M$ is locally convex if and only if at every point there is a local normal vector field, with respect to which the second fundamental form is positive semidefinite.
If there is such local vector field $v_p$ for $p\in M$ then you can construct a convex body $x\in B$
$$\langle x, v_p\rangle\le \langle p, v_p\rangle.$$
For a convex hypersurface (not nesessary smooth) one can choose a smooth local vector field of supporting hyperplanes on $M$. Then choose $v_p$ to be its normal...
I am not sure I understand your construction. Let [B_p=\{x\mid \langle x,v_p\rangle\leq\langle p,v_p\rangle\}.] Is then $B$ defined as the union or intersection of all $B_p$? In the former case why is it convex, and in the latter why is $M$ in its boundary? – user175348 Feb 15 '11 at 21:28
As the intersection; if $v$ is smooth, the condition on the second fundamental form implies that $M$ is on the boundary. That is true only for $p$ in a small set open domain of $M$. – ε-δ Feb 23 '11 at 18:43