# “isotropic” subspaces of a simple Lie algebra

Let $\bf g$ be a finite-dimensional real simple Lie algebra of compact type and let $\left<-,-\right>$ denote the positive-definite inner product induced from the negative of the Killing form. Let $\Omega$ denote the trilinear map defined by $$\Omega(X,Y,Z) = \left<[X,Y],Z\right> .$$ It is easy to see that it is alternating, because of the ad-invariance of the Killing form. Let us call a subspace $S\subset{\bf g}$ isotropic if $\Omega$ vanishes identically when restricted to $S$; that is, if $$\Omega(X,Y,Z) = 0, \forall X,Y,Z \in S.$$

In other words, $S$ is isotropic iff $[S,S] \subset S^\perp$, where ${}^\perp$ means the perpendicular complement relative to the Killing form.

Furthermore we say that an isotropic subspace is maximal if it is not properly contained in an isotropic subspace. It is not hard to show that $S$ is maximal isotropic if and only if $[S,S] = S^\perp$.

The question is how to characterise the maximal isotropic subspaces of $\bf g$.

It is easy to see that the maximally isotropic subalgebras are precisely the Cartan subalgebras, but I am interested in subspaces which are not necessarily subalgebras.

The only examples I know are those for which $S = {\bf k}^\perp$ and ${\bf k} < {\bf g}$ a subalgebra, whence $${\bf g} = {\bf k} \oplus S$$ is a symmetric decomposition corresponding to the compact riemannian symmetric space $G/K$.

Question: Are there any other maximal isotropic subspaces?

-
It seems that Cartan subalgebras (which I agree are maximally isotropic) are a counter-example to the assertion that max isotropic iff [S,S]=S^\perp? –  Fran Burstall Nov 3 '09 at 21:04
(Hi Fran!) I don't think so. I agree that Cartan subalgebras are maximally isotropic subalgebras, but not maximally isotropic subspaces. In other words, they are properly contained in a maximally isotropic subspace which is not a subalgebra. Does this make sense? –  José Figueroa-O'Farrill Nov 4 '09 at 13:41
To make my comment above more precise: Let h < g be a Cartan subalgebra. Then the subspace h + Rx, for any x in g but not in h is still isotropic, but no longer a subalgebra. (We're in the compact case, remember.) –  José Figueroa-O'Farrill Nov 4 '09 at 14:04
Oops. Silly me. I see yr point now. It raises a possibly interesting question though: what are the max isotropic subspaces that contain a Cartan? –  Fran Burstall Nov 4 '09 at 22:44
The question, as stated, is for compact semisimple Lie algebras, but if we relax that condition, then I think that the Borel subalgebras answer the question that you have just posed. Not sure about the answer to your question in positive-definite signature, though; but I do agree that it is interesting. –  José Figueroa-O'Farrill Nov 5 '09 at 10:43

The answer is yes, there are other maximal isotropic subspaces for at least some real Lie algebras of compact type. I thought of a dimension-counting argument that avoids an explicit construction. A subspace of symmetric type is rigid; it has no free parameters under than conjugation in the Lie algebra $\mathbf{g}$. In the dimension sense, there may not be enough of them to contain all of the isotropic subspaces.

For instance, suppose that $\mathbf{g} = \mathrm{su}(3)$. It is 8-dimensional and it has two types of symmetric subspaces of symmetric type, $W_5 = \mathrm{so}(3)^\perp$ which is 5-dimensional, and $W_4 = (\mathrm{su}(2) \oplus \mathrm{u}(1))^\perp$, which is 4-dimensional. Each such isotropic subspace $W_n$ lies in a conjugacy class which is $n$-dimensional.

On the other side, suppose that we build an isotropic space as a flag $V_1 \subset V_2 \subset \cdots \subset V_k$. then there are 8 parameters for $V_1$, 7 for $V_2$, at least 4 for $V_3$ (because the kernel of the map to $\bigwedge^2 V_2^*$ is at least 5-dimensional), and at least 1 for $V_4$ (by the same kernel argument). Then we have to subtract the dimension of the flag variety of $V_4$, which is 6. That makes a moduli space of $V_4$s which is at least 12-dimensional, and possibly exactly that. That is bigger than the manifold of $W_4$s, and it is also bigger than the manifold of $W_5$s times the Grassmannian of 4-planes in each $W_5$.

I haven't checked much larger cases than this one. You could try to refine this argument by explicitly identifying when $V_j$ for some small value of $j$ knocks out some $W_n$ that would have contained it. I conjecture that there are arbitrarily large examples, but I don't know if you need such a refinement to get them.

-

The two dimensional subspace of su(3) generated by the matrices (in the 3*3 defining representation) having ones on the (+-) second and third diagonals seems to satisfy this property. Probably, this construction generalizes to the whole su(n).

-
I am afraid I do not understand what you mean by "(+-) second and third diagonals". In the notation where a_{ij} denotes the entry of the matrix a on the i-th row and j-th column, could you tell me which entries are nonzero? Thanks. –  José Figueroa-O'Farrill Oct 31 '09 at 13:19
I meant upper and lower second and third diagonals. The nonvanishing terms in the first generator are a_{i, i+1} and a{i,i-1} which are ones and all other entries zero. Similarly the second generator consists of ones in the entries a{i, i+2} and a{i, i-2}. –  David Bar Moshe Oct 31 '09 at 14:15
Thanks. I have several comments. First of all, any two-dimensional subspace is automatically isotropic, since the isotropy condition is with respect to a 3-form. The question is, though, if they are maximally isotropic. My second comment is that the two matrices which you suggest, are not in su(3). For one thing, they are real and symmetric. Perhaps you meant changing the signs on the entries below the diagonal, which makes them skewsymmetric and hence in su(3). But in this case, I think that this subspace is not maximally isotropic in su(3). –  José Figueroa-O'Farrill Nov 1 '09 at 6:52