Notion of smoothness for set-valued functions

Question

Is there a way of talking about continuity and smoothness for set valued functions? More precisely, consider $M$ and $N$ topological/smooth manifolds, and let $f$ a function that associates to each point $p\in M$ a subset $f(p) \subset N$ (I haven't made any assumptions on what target sets are allowed, but feel free to discuss cases where some restrictions are required). Is there a meaningful/canonical way of saying that $f$ is continuous or smooth?

For my particular application in mind, $M$ is a smooth manifold, and $f$ associates to each $p\in M$ an open, convex cone inside $T_pM$.

Edit: I should clarify that a convex cone $K$ in some real vector space $V$ is a subset such that

(a) conic: for any $v\in K$ and $r\in \mathbb{R}_+$ $\implies rv \in K$.

(b) convex: for any $v,w\in K$ and $a,b \in \mathbb{R}_+$ $\implies av + bw \in K$

It is open if $K$ is an open subset of $V$, so in particular open cones do not contain the origin.

Answer 1

My idea is that if we want to compare $f(p)$ and $f(q)$ for nearby points $p$ and $q$, then we need to be able to put $f(p)$ and $f(q)$ into the same space. To do this, I'm going to assume that $M$ is a finite-dimensional Riemannian manifold, so that we can make use of a connection $\nabla$ on $M$.

For all $p$, let $f(p)$ be an open cone of $T_p M$. Let $U_p M$ denote the unit sphere in $T_p M$, and define $$g(p) = f(p) \cap U_p M.$$ Since $f(p)$ is a cone, it is the linear span of $g(p)$. Thus, any smoothness on $g$ will apply to $f$ as well.

Let $w \in T_p M$, and define the covariant derivative of $g(p)$ in the direction $w$ by $$\nabla_w g(p) := \{ \nabla_w v \}_{v \in g(p)}.$$

Let $\gamma$ be a smooth curve on $M$ with $\gamma(0) = p$ and $\dot \gamma(0) = w$. Define the parallel transport of $g(p)$ along $\gamma(t)$ by $$\nabla_{\dot\gamma(t)} g(p) := \{ \nabla_{\dot\gamma(t)} v \}_{v \in g(p)}.$$ That is, the parallel transport of the set $g(p)$ is given by transporting each vector in $g(p)$ along the curve $\gamma(t)$.

Now, both $\nabla_{\dot\gamma(t)} g(p)$ and $g(\gamma(t))$ are open subsets of the unit tangent space $U_{\gamma(t)} M$ at the point $\gamma(t)$. If the set-valued function $g$ is to be smooth, then these two sets should be comparable.

Let $\operatorname{Vol}_q$ denote the (finite) volume measure on the unit sphere $U_q M$ at the point $q \in M$, and let $\Delta$ denote the symmetric difference of two sets. Let us say that $g$ is smooth at $p$ in the direction $w$ if $$\operatorname{Vol}_{\gamma(t)} \left( \nabla_{\dot\gamma(t)} g(p) ~\Delta~ g(\gamma(t)) \right) = O(t)$$ for all smooth functions $\gamma$ with $\gamma(0) = p$ and $\dot \gamma(0) = w$. If $g$ is smooth at all points in all directions, then we shall say it is smooth on $M$. Consequently, we shall say that $f$ is smooth if $g$ is smooth.

Answer 2

Three outcomes of a short brainstorming (all inspired by Algebraic Geometry):

1. I second Todd Trimble's comment to your question (it deserves to appear in an answer, so I repeat it here): Synthetic differential geometry gives you a way to talk about "manifolds of subsets", and you have sort of an automatic smoothness built in. But getting into this probably takes you on a long detour...

2. The Algebraic Geometer's way of treating many-valued functions is, very sloppily: Identify functions $X \rightarrow Y$ with their graphs, i.e. subvarieties $\Gamma \subseteq X \times Y$ with $pr_X(\Gamma)=X$ (defined on all of X) and $|pr_X^{-1}(x) \cap \Gamma|\leq 1 \ \forall x \in X$ (single-valued), then drop the second requirement. So maybe you can look at smooth submanifolds of $M \times TM$ subject to the conditions you want?

3. The most reasonable, I would say, and close to the previous: Go to differentiable stacks. For your particular situation you could look at the classifying stack of cones inside tangent bundles: This would be the category fibred in groupoids over the site of differentiable manifolds which has objects $(M,C)$ with $M$ a smooth manifold, $C \subseteq TM$ a submanifold (smooth, except for the tip of the cone) of the total space of the tangent bundle such that each fiber is a cone in $T_x(M)$. Morphisms $(M,C) \rightarrow (M',C')$ should probably be smooth morphisms $f:M \rightarrow M'$ such that $C$ is the pullpack of $C'$ along $Tf$, the differential of $f$. Now a map of differentiable stacks from a manifold $N$ into this stack is the same as a smoothly varying choice of cones in the fibers of the tangent bundle $TN$. The same technique should work with other set-valued maps and is very flexible if you want to modify conditions.

Answer 3

One possible way is to demand that the target set $f(p)$ is a compact set and use Hausdorff distance on the set of compact subsets of $N$ (this aplies to your application by considering the proyective space of $T_p M$).

In the particular aplication, If the cone of $T_p M$ is defined by a subspace of $T_p M$ together with an angle and we assume that continuity implies that the dimension of the cones is constant (this would be given using the Hausdorff distance in the proyective space as above), you can test "smoothness" by considering the map from $M$ to the bundle of grasmannians times angle which is a diferentiable manifold and gives a meaningful way of saying that $f$ is smooth.

However, I believe it should be accepted that the cones "colapse" and decrease its dimension (even if they are never trivial, they can "change their dimension"). For this, the only way I imagine is to consider subsets $(T_pM)^n \times \mathbb{R}$ and consider the cone as the one generated by the $n$ vectors and with angle the value in $\mathbb{R}$ (clearly, there is no canonical way of considering this function, one should say that the map is smooth if there exists a function $g$ to $(T_pM)^n \times \mathbb{R}$ which defines $f$).

Answer 4

Here are some random thoughts. If your cones are polyhedral cones, then maybe you can do something like the following:

Suppose $M$ is $n$ dimensional. Let $Gr_{n-1}(TM)$ be the Grassmannian bundle over $M$ such that the fiber over $p$ is the Grassmannian of $(n-1)$-planes in $T_p M$. Let $Gr_i$ be the product bundle $Gr_{n-1}(TM)^{i}$. Let $Gr$ be some appropriate colimit of the $Gr_i$'s. For example, you could take the colimit of the maps $Gr_i \to Gr_{i+1}$ given by $(P_1,\dots,P_i) \mapsto (P_1,\dots,P_i,P_i)$.

Then define a smooth/continuous/whatever polyhedral-cone-valued-function $f$ to be a smooth/continuous/whatever section $s_f$ of the bundle $Gr$ over $M$. The section $s_f$ assigns to the point $p$ the hyperplanes which form the faces of the polyhedral cone $f(p)$.

I guess this works if you have orderings on the faces of the polyhedral cones. If you don't have orderings, you could take symmetric products of the bundles instead of products.