Question

I've been studying isometric quotients (by compact Lie groups) of compact simply connected homogeneous spaces $G/H$ and their inherited curvature. One of the issues that continually arises is the following problem:

Let $\mathfrak{g} = \mathfrak{h}\oplus\mathfrak{p}$ be an orthogonal decomposition with respect to a Ad G invariant inner product on $\mathfrak{g}$. When is there a $g\in G$ such that $Ad(g)\mathfrak{p}\cap \mathfrak{p} = \{0\}$?

Of course, if the dimension of $\mathfrak{p}$ is too large (equivalently, if the dimension of $\mathfrak{h}$ is too small), then they must intersect nontrivially for dimension reasons. This leads us to the general form of the question:

Let $G$ be a compact Lie group and let $V\subseteq \mathfrak{g} = Lie(G)$ be a vector subspace (but not neccesarily a subalgebra). Assume dim($V)\leq \frac{1}{2}$ dim($\mathfrak{g})$. Is there a $g\in G$ such that $Ad(g)V\cap V = \{0\}$?

Of course, if there is one $g$, an entire neighborhood around $g$ will have this property. But

when is there an open and dense set of $g$ such that $Ad(g)V\cap V=\{0\}$?

(If it helps, feel free to assume that $G$ is simple. I'm pretty sure (but haven't been able to prove) that one can reduce the question to simple $G$ anyway.)

Note that for $G=SU(2)$ or $G=SO(3)$, there is an open and dense set of such $g$. This follows because in this case, $Ad:G\rightarrow SO(\mathfrak{g})$ is surjective and, of course the usual $SO(n)$ action on $\mathbb{R}^n$ acts transitively on k-subspaces. Further, the set of all elements of $SO(3)$ that fix a given line in $\mathbb{R}^3$ is isomorphic to an $SO(2)$, which is codimension 2 in $SO(3)$.

However, for any other $G$, this trick doesn't work as $Ad$ will fail to be surjective.

Thank you in advance for your time and feel free to add more tags as appropriate!

Answer 1

I think that answers to your questions can be obtained along the following lines.

Fix the dimension of $V$ to be $k\leq 1/2\dim V$ and let $n=\dim\mathfrak{g}.$ Condiser the real algebraic Grassmanian variety $X=Gr(k,\mathfrak{g}), \dim X=k(n-k).$ The set $(V_1,V_2): \dim(V_1\cap V_2)\geq 1$ is a Zariski closed subset of $X\times X$ of codimension $n-2k+1$ and for a fixed $V,$ the set $W: \dim(V\cap W)\geq 1$ is a Zariski closed subset $Z$ of $X$ of codimension $n-2k+1$ containing $V.$ Then $G$ is a linear algebraic group over $\mathbb{R}$ that acts on $X$ and the negation of the first question may be restated as follows:

Is the $G$-orbit $O=G\cdot V\subset X$ contained entirely in $Z$?

Note that $O$ is Zariski closed and connected in the real topology (assuming that $G$ is connected). The general answer depends on the stabilizer $G_V$ of $V$ in $G,$ which coincides with the stabilizer of the Lie subalgebra $\langle V\rangle\subset\mathfrak{g}$ generated by $V.$ For example, if $V$ is in the center of $\mathfrak{g}$ then $G_V=G$ and $gV=V$ for each $g\in G$ (and more generally, if $V$ nontrivially intersects the center of $\mathfrak{g}$ then $gV\cap V$ won't reduce to $0$ for any $g$). Some easy cases, such as $\dim V=1,$ may be decided by the dimension count.

As for your second question, it is really equivalent to the first one. The set of $g\in G$ such that $V\cap gV={0}$ is the pre-image under the orbit map $G\to G/G_V\simeq O$ of the Zariski open subset $O'$ of $O$ which is the complement of $O\cap Z.$ If this set is non-empty (yes to the 1st question) then it is open and dense in the real topology (so yes to the 2nd question).