complement of a totally disconnected closed set in the plane

Question

While preparing a course in complex analysis, I stumbled over a remark in Dudziak's book on removable sets, namely that any totally disconnected $K \subset\subset {\mathbb C}$ must have a connected complement; a remark, "that verifying the reader may find one of those exercises in 'mere' point-set topology that is a wee bit frustrating". Out of curiosity I spent an evening with this question. The assertion turned out to be a simple consequence of Theorem 14.2 ("If $x$ and $y$ are separated by the closed set $F$ in the open or closed plane they are separated by a component of $F$.") in an old book "Elements of the Topology of Plane Sets of Points" by M.H.A. Newman, Cambridge 1951. There, the proof is based on an lemma by Alexander (used in his proof of the Jordan-Brouwer separation theorem; Trans. AMS 23, 333-349, 1922), stating that, for disjoint closed sets $F_1$ and $F_2$ in the plane, two points which are connected in the complement of $F_1$ and in the complement of $F_2$ are connected in the complement of $F_1 \cup F_2$. This lemma fails for more general surfaces (Newman gives a counterexample for the torus) and is proved by homological methods. So, here are my questions:

(a) is there a more modern reference to these kind of results (Newman's book uses quite an ideosyncratic terminology and notation);

(b) is there a simpler proof not using Alexander's lemma (or similarily deep results);

(c) how about the connectedness of the complement of a totally disconnected closed set in surfaces (or topological spaces) more general than the plane?

Answer 1

A proof of the statement using knowledge that has withstood the test of time is to cite the Alexander duality theorem, for which you can find multiple modern sources. The relevant form of Alexander duality states that the if $X$ is a compact subset of $S^n$, then the reduced Čech cohomology of $X$ is isomorphic to the reduced homology of its complement in the complementary dimension less 1, dimension $q \leftrightarrow n-q-1$. So the reduced $H_0$ of $\mathbb R^2 \setminus X $, which measures disconnectedness and is equal to that of $S^2 \setminus X$, is isomorphic to the 1-dimensional Čech cohomology of $X$; all but 0-dimensional Čech cohomology is trivial for any totally disconnected space, since there are fine covers with 0-dimensional nerve.

The Čech cohomology of $X$ is the direct limit of the cohomology of the nerve of coverings; the nerve is the simplicial complex whose simplices assert that the elements of the cover indexed by its vertices have a non-empty intersection. Using extension theorems, there is a continuous map from a topological space to the nerve of an open cover. What this translates to for any $X$ such that $\mathbb{R}^2 \setminus X$ is disconnected, you can use that fact to define a map from the nerve of any cover of $X$ to $S^1$ which is not null-homotopic, and even when you refine the cover and pull the map back to the refinement, it's still not nullhomotopic. That is inconsistent with $X$ being totally disconnected.

Intuitively, the Alexander theorem detects linking between $X$ and its complement. Every $q$-cycle $Z^q \subset S^n \setminus X$, when $q \ne 0, n$ is the boundary of a $q+1$-chain $C^{q+1}$ in $S^n$, so the nontriviality of $Z^q$ is captured by the way $C^{q+1}$ intersects $X$. Alexander duality is a very natural way to formalize this idea. Ordinary homology is good enough for the complement of $X$: since it is open, it is locally contractible, and the brand of homology or cohomology is not an issue.

The same reasoning shows that the complement of any totally disconnected subset of any manifold of dimension $n \ge 2$ is connected.

Answer 2

I know this is an old question that already has an excellent answer (although it may apply only to compact sets). However, let me respond to the original question, concerning the following results:

Lemma. If two points $\newcommand{\R}{\mathbb{R}}x,y\in\R^2$ are disconnected from each other by the closed set $F$, then they are disconnected by a connected component of $F$.

It is clear (as noted) that the specific topology of the plane has to be used in some way, as e.g. this is no longer true on the torus. The key is indeed the following.

Janiszewski's Theorem. If two compact sets $A,B\subset S^2$ do not separate $S^2$, and $A\cap B$ is connected, then $A\cup B$ also does not separate $S^2$.

This result goes back to Janiszewski (1912), rather than the article of Alexander you mention. It is key to proving many "obvious" facts in plane topology (in particular it can be used to give a topological proof of the Jordan curve theorem). For example, a quick search found these notes giving a direct proof of Janiczewski's theorem online. I also seem to recall that there is a referene to a topological proof in Pommerenke's "Boundary behaviour of conformal maps", but I don't have the book to hand at the moment.

In any case, it should hopefully be clear that a direct argument can be furnished (e.g. by modifying a curve connecting two given points by replacing certain chords with those of simple closed curves sufficiently close around the two sets), but it is always going to be a bit fiddly.

Remark 1. The result in the form you quoted:

Corollary. If two compact sets $A,B\subset S^2$ do not separate the points $x,y\in S^2$, and $A\cap B$ is connected, then $A\cup B$ also does not separate $x$ and $y$.

follows easily from Janiczewski's theorem (filling in the components of $S^2\setminus A$ that do not contain $x,y$, and similarly for $B$).

Remark 2. A proof of the lemma above (which turns out to be very useful), in slightly different formulation, can be found e.g. in my paper Connected escaping sets of exponential maps, DOI:10.5186/aasfm.2011.3604 where it was given for completeness, although in that sketch the relevance of Janiczewski's theorem is brushed under the carpet in the last line.