MathOverflow will be down for maintenance for approximately 3 hours, starting Monday evening (06/24/2013) at approximately 9:00 PM Eastern time (UTC-4).

2 added 36 characters in body

Okay, now I think I understand your question. This is the question I will answer:

• Question: Let $X$ be a connected $4$-dimensional subspace of $SE(3)$ that contains $SO(3)$. Is it possible for $X \setminus SO(3)$ to be connected? Disconnected?

The answer to both questions is yes. So there is no Jordan separation theorem for $4$-dimensional subspaces of $SE(3)$ containing $SO(3)$.

Observation 1: As a space, $SE(3)$ is just the cartesian product of $SO(3)$ with $\mathbb R^3$. Explicitly, we will think of $SE(3)$ as the set $SO(3) \times \mathbb R^3$.

Observation 2: If $X := SO(3) \times \mathbb R$ embeds in $SE(3)$, therefore $SO(3) \times \{0\}$ disconnects it.

Observation 3: If $X := SO(3) \times S^1$, where $S^1 = \{ x \in \mathbb R^2 : |x|=1\}$, then the map $X \to SO(3) \times \mathbb R^3$ given by $(p,x) \longmapsto (p,x,0)$ is an embedding. In particular, $X \setminus (SO(3) \times \{1\})$ is connected.

So the answer to both your questions is yes.

I'd like to suggest looking at the proof of the generalized Jordan-Brouwer theorem in Guillemin and Pollack, or perhaps in an algebraic topology textbook like Bredon's. This will give you a very flexible set of tools that will let you know quite generally when you can expect a separation theorem, and when you can't.

Notice: my answer had nothing to do with the fact that $SO(3)$ has a non-trivial fundamental group, or whether or not it is orientable. The key part of the construction is that $SO(3)$ has co-dimension at least $2$ (And actually co-dimension $3$) in $SE(3)$.

1

Okay, now I think I understand your question. This is the question I will answer:

• Question: Let $X$ be a connected $4$-dimensional subspace of $SE(3)$ that contains $SO(3)$. Is it possible for $X \setminus SO(3)$ to be connected? Disconnected?

The answer to both questions is yes. So there is no Jordan separation theorem for $4$-dimensional subspaces of $SE(3)$ containing $SO(3)$.

Observation 1: As a space, $SE(3)$ is just the cartesian product of $SO(3)$ with $\mathbb R^3$. Explicitly, we will think of $SE(3)$ as the set $SO(3) \times \mathbb R^3$.

Observation 2: If $X := SO(3) \times \mathbb R$ embeds in $SE(3)$, therefore $SO(3) \times \{0\}$ disconnects it.

Observation 3: If $X := SO(3) \times S^1$, where $S^1 = \{ x \in \mathbb R^2 : |x|=1\}$, then the map $X \to SO(3) \times \mathbb R^3$ given by $(p,x) \longmapsto (p,x,0)$ is an embedding. In particular, $X \setminus (SO(3) \times \{1\})$ is connected.

So the answer to both your questions is yes.

I'd like to suggest looking at the proof of the generalized Jordan-Brouwer theorem in Guillemin and Pollack, or perhaps in an algebraic topology textbook like Bredon's. This will give you a very flexible set of tools that will let you know quite generally when you can expect a separation theorem, and when you can't.

Notice: my answer had nothing to do with the fact that $SO(3)$ has a non-trivial fundamental group. The key part of the construction is that $SO(3)$ has co-dimension at least $2$ (And actually co-dimension $3$) in $SE(3)$.