Any function $f\colon A\to B$ defines a triple adjunction $f_*\dashv f^{-1}\dashv f_!$ between the powersets $\mathcal{P}(A)$ and $\mathcal{P}(B)$, where \begin{align*} f_*(U) &:= \{b\in B\ |\ \text{there exists $a\in U$ such that $f(a)=b$}\}\\ f^{-1}(V) &:= \{a\in A\ |\ f(a)\in V\}\\ f_!(U) &:= \{b\in B\ |\ f^{-1}(b)\subset U\} \end{align*} with $U\in\mathcal{P}(A)$ and $V\in\mathcal{P}(B)$. Now, we may define open, closed, and continuous maps using these:
- A map $f$ is open if $f_*$ sends opens to opens.
- A map $f$ is continuous if $f^{-1}$ sends opens to opens.
- A map $f$ is closed if $f_!$ sends opens to opens (see the proof by მამუკა here).
We could now repeat this procedure with relations, although this time the triple adjunction breaks down into two adjunctions: any relation $R\colon A ⇸ B$ defines two adjunctions $R_*\dashv R_{-1}$ and $R^{-1}\dashv R_!$, where \begin{align*} R_*(U) &:= \{b\in B\ |\ \text{there exists $a\in U$ such that $b\in R(a)$}\}\\ R_{-1}(V) &:= \{a\in A\ |\ R(a)\subset V\}\\ R^{-1}(V) &:= \{a\in A\ |\ R(a)\cap V\neq\emptyset\}\\ R_!(U) &:= \{b\in B\ |\ R^{-1}(b)\subset U\} \end{align*} with $U\in\mathcal{P}(A)$ and $V\in\mathcal{P}(B)$.
Note: A nice fact here is that $R_{-1}=R^{-1}$ iff $R$ is total and functional, i.e. $R^{-1}$ and $R_{-1}$ coincide precisely if $R$ comes from a function.
Mimicking the situation for functions, we could now make the following definitions:
- A relation $R$ is open if $R_*$ sends opens to opens.
- A relation $R$ is strongly continuous if $R_{-1}$ sends opens to opens.
- A relation $R$ is weakly continuous if $R^{-1}$ sends opens to opens.
- A relation $R$ is closed if $R_!$ sends opens to opens (A very similar argument to the one given by მამუკა for functions shows that this is the same as asking $R_*$ to send closed sets to closed sets).
There is, however, an issue: continuous maps can be equivalently defined as those $f$ for which $f$ sends closed sets to closed sets, which follows from the equality $A\setminus f^{-1}(V)=f^{-1}(B\setminus V)$. Now, this equality doesn't need to hold for either $R_{-1}$ or $R^{-1}$, as we have \begin{align*} R_{-1}(B\setminus V) &= \{a\in A\ |\ R(a)\subset B\setminus V\},\\ A\setminus R_{-1}(V) &= \{a\in A\ |\ R(a)\not\subset V\},\\ R^{-1}(B\setminus V) &= \{a\in A\ |\ R(a)\setminus V\neq\emptyset\},\\ A\setminus R^{-1}(V) &= \{a\in A\ |\ R(a)\cap V=\emptyset\}. \end{align*} Considering also relations $R$ for which $R^{-1}$ or $R_{-1}$ send closed sets to closed sets thus leads to a total of four different definitions of continuity for relations using this approach. (Edit: the situation turns out to be way better if $R$ is total; see below)
Edit: I've since found two references which develop the above ideas further:
- The first one is Clementino–Tholen's A characterization of the Vietoris topology [PDF], which develops further the theory of open and closed relations, in particular proving the following theorem:
Theorem (Clementino–Tholen). Let $R\colon X\times Y\to\{0,1\}$ be a relation from $X$ to $Y$, and let $\mathcal{P}^{-}(X)$, $\mathcal{P}^{+}(X)$, and $\mathcal{P}(X)$ denote the lower Vietoris, upper Vietoris, and Vietoris topology on $\mathcal{P}(X)$. The following conditions are equivalent:
- The relation $R$ is open.
- The map $R^{-1}\colon\mathcal{P}^{-}(Y)\to\mathcal{P}^{-}(X)$ is continuous.
- The adjunct $R\colon Y\to\mathcal{P}^{-}(X)$ of $R$ is continuous.
- We have $R^{-1}(\overline{V})\subset\overline{R^{-1}(V)}$ for each $V\in\mathcal{P}(Y)$, where $\overline{S}$ denotes the closure of a set $S$.
Similarly, the following conditions are also equivalent:
- The relation $R$ is closed.
- The map $R^{-1}\colon\mathcal{P}^{+}(Y)\to\mathcal{P}^{+}(X)$ is continuous.
- The adjunct $R\colon Y\to\mathcal{P}^{+}(X)$ of $R$ is continuous.
- We have $R_{*}(\overline{U})\supset\overline{R_{*}(U)}$ for each $U\in\mathcal{P}(X)$.
- The second one is Klein–Thompson, Theory of Correspondences [Link], which develops the theory of "weakly/strongly continuity" as defined above for total relations (the situation is a little complicated for general relations):
- First, it seems that the definitions of continuity for relations commonly used in practice are instead the following:
- lower semicontinuity, called "weak continuity" above;
- upper semicontinuity, the property that $R^{-1}$ sends closed sets to closed sets.
- Now, the four definitions situation for continuity of relations described above gets better for total relations: when $R$ is total Proposition 6.3.5 there notes that we have the following equalities: \begin{align*} R_{-1}(B\setminus V) &= A\setminus R^{-1}(V),\\ R^{-1}(B\setminus V) &= A\setminus R_{-1}(V). \end{align*} As a consequence, $R^{-1}$ preserves opens (resp. closed sets) iff $R_{-1}$ preserves closed sets (resp. opens)!
- Lastly we have the following result (Theorems 7.1.4 and 7.1.7) which gives equivalent conditions for $R$ to be continuous:
- First, it seems that the definitions of continuity for relations commonly used in practice are instead the following:
Theorem (Klein–Thompson). If $R$ is total, then the following conditions are equivalent:
- The relation $R$ is upper semicontinuous, i.e. $R^{-1}$ sends closed sets to closed sets.
- The adjunct $R\colon X\to\mathcal{P}^{+}(Y)$ of $R$ is continuous.
- The function $R_{-1}$ sends opens to opens.
- For every $x\in X$, every net $(x_n)_{n\in D}$ in $X$ converging to $x$, and every open set $V$ of $Y$ with $f(x)\subset V$, we have $f(x_n)\subset V$ for sufficiently large $n$.
Similarly, the following conditions are equivalent:
- The relation $R$ is lower semicontinuous, i.e. $R^{-1}$ sends opens to opens.
- The adjunct $R\colon X\to\mathcal{P}^{-}(Y)$ of $R$ is continuous.
- The function $R_{-1}$ sends closed sets to closed sets.
- For every $x\in X$, every net $(x_n)_{n\in D}$ in $X$ converging to $x$, and every open set $V$ of $Y$ with $f(x)\cap V\neq\emptyset$, we have $f(x_n)\cap V\neq\emptyset$ for sufficiently large $n$.
Lastly Klein–Thompson also give two results comparing continuity/closedness of relations in the above senses to $R\subset X\times Y$ being a closed set in the product topology:
Theorem 7.1.15. If $Y$ is regular, $R$ is upper semicontinuous, and $R_*(x)$ is closed for each $x\in X$, then $R\subset X\times Y$ is closed with respect to the product topology.
Theorem 7.1.16. If $Y$ is compact and $R$ is a closed relation in that $R_*$ maps closed sets to closed sets, then $R\subset X\times Y$ is closed with respect to the product topology.
(Theorem 7.1.16 fails if $Y$ is allowed to be noncompact; see Example 7.1.17 there)
Some remarks on equivalences relation.
One reason for us to care about relations being upper/lower semicontinuous, open or closed is because of quotient spaces: when an equivalence relation $\sim$ on a topological space $X$ satisfies some of these properties, we in fact get a bunch of nice facts about the quotient $X/\mathord{\sim}$ being well-behaved. For instance, here's a result from Clementino–Tholen:
Let $R$ be an equivalence relation on $X$. The following conditions are equivalent:
- The relation $R$ is closed (resp. open)
- The quotient map $\pi\colon X\twoheadrightarrow X/\mathord{\sim}_R$ is closed (resp. open)
- The inclusion map $\iota_{X/\mathord{\sim}_R}\colon X/\mathord{\sim}_R\to\mathcal{P}^{+}(X)$ (resp. to $\mathcal{P}^{-}$(X)) is continuous.
Here are a couple of other results I found:
- See Daniele Zuddas's answer
- (Tu An Introduction to Manifolds, Theorem 7.7.) If $R$ is an open equivalence relation on $X$, then $X/\mathord{\sim}_R$ is Hausdorff iff $R\subset X\times Y$ is closed in the product topology.
- (Tu An Introduction to Manifolds, Corollary 7.10.) If $R$ is open and $X$ second-countable, then $X/\mathord{\sim}_R$ is second-countable.
- Daverman's Decompositions of Manifolds has a bunch of other results, too.
Comparisons with definitions in other answers. Here's a comparison of the notions above with some of the ones in the other answers:
- Putting topologies on powersets. As seen above, continuity, openness and closedness of relations in the above senses correspond to asking the associated functions $X\to\mathcal{P}(Y)$ or $Y\to\mathcal{P}(X)$ to be continuous with respect to the Vietoris topologies. (There are other interesting powerset topologies to consider besides the Vietoris ones, though, like the Fell topology (or maybe the Alexandroff topology; I don't know if it is different from the Vietoris ones))
- Closedness in the product topology. Klein–Thompson's Theorems 7.1.15 and 7.1.16 relate closedness of $R$ in the product topology to $R$ being a closed relation. The two notions are distinct in general, however.
- Eric Wofsey's approach via nets. As Klein–Thompson's Theorem 7.1.7 shows, a total relation $R$ satisfies Eric's criterion iff it is lower semicontinuous, i.e. iff $R^{-1}$ sends opens to opens.
- Lehs's upper/lower hemicontinuity. These are equivalent (I think!) to asking that $R_{-1}$ and $R^{-1}$ preserve opens!