Let $(M,g)$ be a compact smooth simply connected Riemannian manifold with a smooth boundary. Assume also that $(M,g)$ does not have any conjugate pairs of points. Let $\Gamma \subset \partial M$ be a smooth connected subset of the boundary that is strictly concave in the sense of the second fundamental form. Prove that there can be no inextendible geodesic in $M$ with both end points on $\Gamma$.
1 Answer
Here is a proof that works, I think—although it's probably a bit clumsier than necessary. The short version is that the geodesics in $M \setminus \partial M$ are minimizing. This means we can't connect two points $p,q \in \Gamma$ by a geodesic in $M$, because there is a strictly shorter curve inside $\partial M$.
Now for the long version: we start with the following claim.
Claim. Let $x \in M$. Then $\operatorname{exp}_x: \operatorname{exp}_x^{1}(M \setminus \partial M) \subset T_x M \to M \setminus \partial M$ is a diffeomorphism.
This is proved by essentially following the proof of the Cartan–Hadamard theorem; one notable difference is that the domain of definition is restricted because we are on a manifold with boundary. The argument essentially goes as follows: as $M$ has no conjugate points, the exponential map is a local diffeomorphism. In fact it is a covering map, and $M$ being simply connected, it must be a diffeomorphism.
We rely on the claim for the following corollary. It implies that any two points in $M \setminus \partial M$ are connected by a unique geodesic, and therefore all geodesics in $M \setminus \partial M$ are minimizing.
All this preamble being completed, let's get to the proof.
Proof. We argue by contradiction. Let $p,q \in \Gamma \subset \partial M$ be two points in the boundary, connected by some geodesic $\gamma: [0,L] \to M$ in the interior of $M$, with only its endpoints on the boundary. Let $\eta: [0,d] \to \partial M$ be the curve realizing the distance $d < L$ between the two; as $\Gamma$ is strictly concave, this must lie inside the boundary.
By the (corollary to) the claim, the geodesic $\gamma$ must be minimizing, at least for all times $t \in (0,L)$. To obtain a contradiction, take $\epsilon > 0$ small enough that $d + 5 \epsilon < L$. Consider the curve connecting the two points $\gamma(\epsilon)$ and $\gamma(L\epsilon)$ by:
 first going from $\gamma(\epsilon)$ to $\gamma(0) = p$ via $\gamma^{1}$;
 then from $p$ to $q$ via $\eta$;
 finally from $q = \gamma(L)$ to $\gamma(L\epsilon)$ via $\gamma^{1}$ again.
This has length $d + 2 \epsilon$, whereas going along $\gamma$ would give length $L  2 \epsilon$.
We chose $\epsilon$ so small that $d + 3 \epsilon < L  2\epsilon$. We use the extra wiggle room to move $\eta$ into the interior, to a curve $\eta'$ with length at most $d + \epsilon$. The curve resulting by pasting $\eta'$ into the $\eta$portion has length at most $d + 3 \epsilon < L  2\epsilon$, which is absurd because $\gamma$ is minimizing.

$\begingroup$ I was looking at this proof today and do not understand why $d$ must be less than $L$. Could you justify this rigorously? Definition of convexity is local (based on second fundamental form) but here the d<L is a more global statement $\endgroup$– AliMay 21, 2023 at 11:49