"isotropic" subspaces of a simple Lie algebra

Question

Let $\bf g$ be a finite-dimensional real simple Lie algebra of compact type and let $\left<-,-\right>$ denote the positive-definite inner product induced from the negative of the Killing form. Let $\Omega$ denote the trilinear map defined by $$\Omega(X,Y,Z) = \left<[X,Y],Z\right> .$$ It is easy to see that it is alternating, because of the ad-invariance of the Killing form. Let us call a subspace $S\subset{\bf g}$ isotropic if $\Omega$ vanishes identically when restricted to $S$; that is, if $$\Omega(X,Y,Z) = 0, \forall X,Y,Z \in S.$$

In other words, $S$ is isotropic iff $[S,S] \subset S^\perp$, where ${}^\perp$ means the perpendicular complement relative to the Killing form.

Furthermore we say that an isotropic subspace is maximal if it is not properly contained in an isotropic subspace. It is not hard to show that $S$ is maximal isotropic if and only if $[S,S] = S^\perp$.

The question is how to characterise the maximal isotropic subspaces of $\bf g$.

It is easy to see that the maximally isotropic subalgebras are precisely the Cartan subalgebras, but I am interested in subspaces which are not necessarily subalgebras.

The only examples I know are those for which $S = {\bf k}^\perp$ and ${\bf k} < {\bf g}$ a subalgebra, whence $${\bf g} = {\bf k} \oplus S$$ is a symmetric decomposition corresponding to the compact riemannian symmetric space $G/K$.

Question: Are there any other maximal isotropic subspaces?

Answer 1

The answer is yes, there are other maximal isotropic subspaces for at least some real Lie algebras of compact type. I thought of a dimension-counting argument that avoids an explicit construction. A subspace of symmetric type is rigid; it has no free parameters under than conjugation in the Lie algebra $\mathbf{g}$. In the dimension sense, there may not be enough of them to contain all of the isotropic subspaces.

For instance, suppose that $\mathbf{g} = \mathrm{su}(3)$. It is 8-dimensional and it has two types of symmetric subspaces of symmetric type, $W_5 = \mathrm{so}(3)^\perp$ which is 5-dimensional, and $W_4 = (\mathrm{su}(2) \oplus \mathrm{u}(1))^\perp$, which is 4-dimensional. Each such isotropic subspace $W_n$ lies in a conjugacy class which is $n$-dimensional.

On the other side, suppose that we build an isotropic space as a flag $V_1 \subset V_2 \subset \cdots \subset V_k$. then there are 8 parameters for $V_1$, 7 for $V_2$, at least 4 for $V_3$ (because the kernel of the map to $\bigwedge^2 V_2^*$ is at least 5-dimensional), and at least 1 for $V_4$ (by the same kernel argument). Then we have to subtract the dimension of the flag variety of $V_4$, which is 6. That makes a moduli space of $V_4$s which is at least 12-dimensional, and possibly exactly that. That is bigger than the manifold of $W_4$s, and it is also bigger than the manifold of $W_5$s times the Grassmannian of 4-planes in each $W_5$.

I haven't checked much larger cases than this one. You could try to refine this argument by explicitly identifying when $V_j$ for some small value of $j$ knocks out some $W_n$ that would have contained it. I conjecture that there are arbitrarily large examples, but I don't know if you need such a refinement to get them.

Answer 2

The two dimensional subspace of su(3) generated by the matrices (in the 3*3 defining representation) having ones on the (+-) second and third diagonals seems to satisfy this property. Probably, this construction generalizes to the whole su(n).