Initially, I want to find a function which is well-defined in the whole domain $\overline{\mathbb{S}}^n_+$ and strictly convex in the interior. So there is impossible. But it is true on the spherical cap due to your example. Many thanks.

Thanks for the detail. Since the function $f=\frac{1}{x_{n+1}}$ has no definition on the equator (the boundary of hemisphere)? When restricting it to the boundary of $\mathbb{S}^n_+$, how to see it is strict convexity on the equator?

Thanks, Yes, it is equivalent to the function is strictly convex when restricted it to any geodesic $\gamma(t)$ be strictly convex, $\frac{d}{dt^2} f(\gamma(t))\geq c_0>0$, for any geodesic $\gamma(t)\subset \mathbb{S}^n_+$. But the circle $\sigma_{\epsilon}$ of radius $\frac{\pi}{2}$ in the hemisphere is not a geodesic? how does the contradiction coming out? Thanks for more explanation.

I wonder whether the normal covector of $T^*M_{\partial M}$ (view it as submanifold and boundary of $T^*M$ ) has some relation with the normal covector of $\partial M$ ?

Thanks for comments. You mean view $\partial T^*(M)$ as $T^*(M)|_{\partial M}$ bundle over $\partial M$ with same $n$ dimensional fiber as $T^*(M)$ ? Is the normal vector field of $T^*(M)|_{\partial M}$ be induced from $\nu\in T_{\partial M}M$? say, for a section $s\in T^*(M)$ of cotangent bundle, is the normal vector field at point $(x,s(x))\in T^*(M)|_{\partial M} $( here, $x\in\partial M$) in $T^*(M)$ be $(\nu_x,\nu_x \lrcorner \nabla s)$?