Question

Let $G$ be a compact group and $\rho: G \to End(U)$ its linear representation in a finite dimensional vector space $U$. Fix $V \subset U$ - a subspace invariant under $\rho(G)$. Then it is well known that $V$ has some invairant complement.

What are the sufficient conditions (on $G$, $\rho$ or $V$) to ensure this complement is unique?

Stated another way, starting with any scalar product on $U$, invariant complements to $V$ can be found by averaging it w.r.t. to a Haar measure on $G$ and taking $V^\perp$. In this case my question becomes

What are the sufficient conditions that for any choice of the initial scalar product, the resulting $V^\perp$ is unique.

There are examples when the complement is unique. For example, consider $\mathbb{R}^3$ and the action of $SO(2)$ given by the rotation around the $z$-axis. Then if $V$ is a $z$-axis it has a unique complement.

Particular example I am trying to understand is the following: consider the group $U(n)$ acting on $\mathbb{R}^{2n}$ in a standard way. This action induces a representation $\rho$ in $U=Hom(\mathbb{R}^{2n} \wedge \mathbb{R}^{2n}, \mathbb{R}^{2n})$ given by $$ (g S) (u\wedge v) = g^{-1}S (gu \wedge gv) $$ Consider a map $A: Hom(\mathbb{R}^{2n}, \mathfrak{u}(n)) \to Hom(\mathbb{R}^{2n} \wedge \mathbb{R}^{2n}, \mathbb{R}^{2n})$ defined by $$ (AS)(u \wedge v) = S(u)v - S(v)u $$ and define $V = A(Hom(\mathbb{R}^{2n}, \mathfrak{u}(n)))$.

For what $n$, there is a unique invariant complement to $V$?

Such construction appears in the Cartan's method of equivalence. Each $U(n)$-invariant complement to $V$ corresponds to a certain $U(n)$-invariant linear connection.

Thanks,

Answer 1

This happens if and only if $U$ is expressible as a sum of pairwise non-isomorphic irreducible $G$-modules. If, for example $U \cong V \oplus V$ for an irreducible $G$-module $V,$ then the natural invariant submodule $ \{(v,0): v \in V \}$ has at least two complements: one is the natural choice $\{(0,v): v \in V \}.$ Another is $\{(v,v): v \in V \}.$ On the other hand, if $U \cong \bigoplus_{i=1}^{n} V_i,$ where $V_i \not \cong V_j$ for $i \neq j,$ then the only $G$-submodules of $U$ are of the form $V_I = \bigoplus_{i \in I} V_i$ for a subset $I$ of $\{1,2,\ldots ,n\},$ and each such $G$-submodule $V_I$ has the unique complement $V_{I^{\prime}},$ where $I^{\prime} = \{ 1,2,\ldots ,n \} \backslash I.$

Answer 2

As other answers indicated, the answer to yr first two questions is that a $G$-invariant complement to $V\subset U$ is unique iff $V$ and $U/V$ do not contain $G$-isomorphic invariant subspaces. You can prove it quite easily with Schur´s lemma.

The answer to your 3rd question is that for no $n$ there is a unique invariant complement to your $V$. That is, both your $V$ and $U/V$ contain $U(n)$-isomorphic subspaces.

In order to show this it is easier (for me) to complexify your representations. Denote by $W$ the usual representation of $U(n)$ on $\mathbb C^n$ (multiplication of column vectors by unitary matrices on the left). Then it is quite easy to verify the following isomorphisms of $U(n)$-representations:

${\mathbb R}^{2n} \otimes {\mathbb C} = W \oplus W^*$,

$\Lambda^2({\mathbb R}^{2n})\otimes {\mathbb C}=\Lambda^2 W \oplus \Lambda^2 W^* \oplus (W \otimes W^*)$

${\mathfrak u}(n)\otimes {\mathbb C}=W \otimes W^*.$

Also, you can show that your map $A$ is injective (it is in fact injective on all of $Hom({\mathbb R}^{2n},\mathfrak o(2n))$, which is a standard fact in the Cartan theory, sometimes called the $S_3$ lemma).

So you get the following $U(n)$-decomposition of $V$

$V\otimes{\mathbb C}=Hom({\mathbb R}^{2n},\mathfrak u(n))\otimes{\mathbb C}=(W \oplus W^* ) \otimes ( W \otimes W^* )=$

$=W\otimes W \otimes W^* + conj.$

(the $+conj $ means you need to add to previous summands their conjugate, or dual, which in this case is just $W^* \otimes W^* \otimes W$).

Next

$U\otimes{\mathbb C}=Hom(\Lambda^2({\mathbb R}^{2n}), {\mathbb R}^{2n})\otimes{\mathbb C}=(\Lambda^2 W \oplus \Lambda^2 W^* \oplus (W \otimes W^* ))\otimes ( W \oplus W^* ), $

so "substracting" $V \otimes{\mathbb C}$ you get

$(U/V)\otimes{\mathbb C}=(\Lambda^2 W \oplus \Lambda^2 W^* )\otimes ( W \oplus W^* ) =(\Lambda^2 W \otimes W) \oplus (\Lambda^2 W \otimes W^* )+conj.$

Now since $W\otimes W = \Lambda^2 W \oplus S^2 W$ you can see clearly that both $V$ and $U/V$ (or rather their complexificacions) contain subspaces isomorphic to $\Lambda^2 W\otimes W^* $, so that an invariant complement to $V$ is not unique.

Note that I didn't bother to decompose the spaces into irreducibles since it was not necessary for answering yr question. The common summand $\Lambda^2 W\otimes W^* $ contains in fact a summand isomorphic to $W$ ( it is given by the contraction mapping $\Lambda^2 W\otimes W^*\to W $), which means that $V$ and $U/V$ both have an invariant subspace ismorphic to the standard representation on ${\mathbb R}^{2n}$. Probably one can see it directly without the calculation above but I didn't try.

Answer 3

Geoff's answer (despite the notation) gives the conditions on the ambient subspace to have all subspaces having a unique complement; the condition on $V$ is that there are no non-zero equivariant maps $U/V\to V$, since Gavin chosen one complement, the others are in bijection with such maps. This is the same as saying $V$ is a sum of isotopic components, that is, that there is no simple which shows up both in $V$ and $U/V$.