Invariants of exterior powers

Question

Let $\mathfrak{g}$ be the Lie algebra of $GL(n,\mathbb{R})$. Let $\theta(X) = - X^T$ be the Cartan involution on $\mathfrak{g}$; it induces decomposition as $\mathfrak{g} = \mathfrak{k} \oplus \mathfrak{p}$ of $+1$-eigenspace $\mathfrak{k}$ and $-1$ eigenspace $\mathfrak{p}$. Then $\mathfrak{k}$ is the Lie algebra of $K = O(n)$. I want to compute the dimensions of $K$-invariants $\textrm{Hom}_K(\wedge^q \mathfrak{p}, \mathbb{C})$, which, I suppose, is equal to $(W/ \mathfrak{k} W)^*$ where $W = \wedge^q \mathfrak{p}$, where $\mathfrak{p}$ is viewed as $\mathfrak{k}$-module by adjoint-representation. Could you someone point a way further?

Answer 1

To offer a slightly more geometric viewpoint on the same, the space $\bigoplus_q \mathop{\mathrm{Hom}}_K(\Lambda^q(\mathfrak{p}),\mathbb{C})$, which is the direct sum of all spaces you are considering, is the cohomology of the Lie algebra $\mathfrak{gl}_n$ relative to its subalgebra $\mathfrak{o}_n$, which, by a standard argument, is the same as $H^*(U(n)/O(n))$. It is well known (it really is; if you need some reference, see, for example, page 28 of lecture notes of Haefliger - http://www1.mat.uniroma1.it/people/piazza/HaeDC.pdf) to be the exterior algebra generated by elements $\xi_{4k-3}$, for all $k$ such that $2k-1\le n$. In fact, $\xi_{d}$ is the anti-symmetrisation of the trace of product of $d$ matrices. This confirms the very nice answer given by Abdelmalek Abdesselam.

Answer 2

You are basically looking for trivial $O(\mathbb{C}^n)$ representations contained in $\Lambda^k \mathfrak{p}^*.$ Since your representation $\mathfrak{p}$ is the space of symmetric matrices, you can find the whole decomposition into irreducibles of $\Lambda^k S^2 \mathbb{C}^n$ as a special case of plethysm for the Lie group $O(\mathbb{C}^n).$ In fact, this very special case of plethysm was already dealt with on this site for the group $GL(\mathbb{C}^n)$ in (at least) two questions:

Known decomposition of $\bigwedge^k Sym^d \mathbb C^n$ in special cases?

What is known about this plethysm?

Please note, that irreducible $GL(\mathbb{C}^n)$ representations decompose further under restriction to $O(\mathbb{C}^n)$ -- see e.g. Symmetry, Representations, and Invariants by Nolan Wallach and Roe Goodman.

To check whether you get correct decompositions and/or multiplicities you can use mathematical software. See e.g.:

Symmetric powers of Schur polynomials

I've whipped up some Sage code which however seems to be quite slow: https://cocalc.com/share/25e624ba-9091-484f-af2e-71deb7120f58/trivial%20reps%20in%20exterior%20algebra.sagews?viewer=share

Answer 3

Edit: I misread the question and understood $\mathfrak{p}$ as corresponding to antisymetric instead of symmetric matrices. So what immediately follows answers a different question. See the end of this post for a complete solution of the OP's case.

---

I don't know what you mean exactly by "compute the invariants": perhaps finding an explicit linear basis? Essentially $\wedge^q\mathfrak{p}$ is the same as the antisymmetric plethysm $\wedge^q(\wedge^2(\mathbb{R}^n))$. If I remember correctly, this kind of objects appears in relation to the Hodge conjecture for certain Abelian varieties as in work by Ken Ribet and Fumio Hazama. Namely, in some particular situations, the invariant elements in such a plethysm are the Hodge classes in the cohomology of the variety. See in particular the article "Branching rules and Hodge cycles on certain Abelian varieties" in AJM 1988. There could be some useful tools for your question in that literature.

As for a linearly generating set of invariants, it is easy to construct as follows. Let me call a Wick contraction any set partition $P$ of $\{1,2,\ldots, 2q\}$ with blocks of size two. Let me denote by $P_{\ast}$ the special case $$ \{\{1,2\},\{3,4\},\ldots,\{2q-1,2q\}\}\ . $$ For indices $i,j$ in $\{1,\ldots,n\}$, let me use the Kronecker delta notation $\delta_{i,j}$ for the indicator function of the condition $i=j$. For a Wick contraction $P$ and a collection of $2q$ indices $i_1,\ldots,i_{2q}$ in $\{1,\ldots,n\}$, let me write $$ A(P)_{i_1,i_2,\ldots,i_{2q}}=\prod_{\{\alpha,\beta\}\in P} \delta_{i_{\alpha},i_{\beta}}\ . $$ One can view $\wedge^q(\wedge^2(\mathbb{R}^n))$ as the real vector space of all arrays $T=(T_{i_1,\ldots,i_{2q}})_{i_1,\ldots,i_{2q}\in\{1,\ldots,n\}}$ which change sign if one exchanges indices within a block of $P_{\ast}$ or if one rigidly exchanges two blocks. Now define the linear forms $L_P$ indexed by Wick contractions given by $$ L_P(T)=\sum_{i_1,\ldots,i_{2q}\in\{1,\ldots,n\}} A(P)_{i_1,i_2,\ldots,i_{2q}} \ T_{i_1,i_2,\ldots,i_{2q}}\ . $$ By the first fundamental theorem of 19th century invariant theory for $K=O(n)$, these form a linearly generating collection for ${\rm Hom}_K(\wedge^q\mathfrak{p},\mathbb{C})$. Of course one can restrict to $P$'s which are "transverse" to $P_{\ast}$, i.e. , such that $P\cap P_{\ast}=\varnothing$. I don't immediately see how to prune the remaining collection to get a basis, since I expect tons of linear relations. The stable case $n\ge 2q$ might be easier to analyze first.

---

In the OP's case the relevant space is $\Gamma:=\wedge^q({\rm Sym}^2(\mathbb{R}^n))$. The description of a generating set of invariants is exactly the same. The only thing that changes is the identification of $\Gamma$ with a space of arrays $T$. In this situation $T_{i_1,\ldots,i_{2q}}$ must stay the same if two indices in the same block of $P_{\ast}$ are exchanged and it must change sign if two blocks are exchanged. Also the transversality condition can be dropped.

Note that one can reduce the generating set a bit as follows. Instead of Wick contractions (aka perfect matchings) $P$ one can index invariants by integer partitions $\lambda$ of $q$. If one takes $T=A^{(1)}\wedge\cdots\wedge A^{(q)}$ for a bunch of symmetric matrices $A^{(1)},\ldots,A^{(q)}$ then the invariant encoded by $\lambda$ is the complete antisymmetrization of a "manual polarization" of $$ \prod_{a}\ {\rm tr}(A^{\lambda_a})\ . $$ By "manual polarization" I mean replace each $A$ factor by an $A^{(k)}$ so everyone is used exactly once. If one thinks of $P$'s as fixed-point-free involutions, take the cycle type of the composition $P\circ P_{\ast}$ and divide all the cycle lengths by two (extra care needed if lack of transversality). That gives you the partition $\lambda$. It is easy to see that the invariant $L_{\lambda}$ vanishes unless all the parts $\lambda_a$ are congruent to 1 mod 4. It also vanishes if there are repeated (odd) parts. Indeed if one does a cyclic permutation along a trace of length $\lambda_a$ one gets the identity $L_{\lambda}=(-1)^{\lambda_a-1}L_{\lambda}$. So an even part kills the invariant. For $\lambda_a$ congruent to 3 mod 4 one can do a "mirror symmetry" of the cycle which fixes a vertex. This involves $(\lambda_a-1)/2$ disjoint transpositions which is an odd number. If there are two cycles of length $\lambda_a$ then exchanging them gives $(-1)^{\lambda_a^2}=-1$ since we reduced the discussion to odd parts.

So your space of invariants $(\wedge^q({\rm Sym}^2(\mathbb{R}^n)))^{O(n)}$ is of dimension 1 for $q=1$. Then it vanishes for $q=2,3,4$. I suspect it has dimension 1 for $q=5$ and $n$ not too small. By the above it has at most that dimension. Showing equality means showing that the antisymmetrization of $$ {\rm tr}(A^{(1)}A^{(2)}A^{(3)}A^{(4)}A^{(5)}) $$
is not identically zero.

Let $N(q)$ be the number of integer partitions $\lambda$ of $q$ with distinct parts that are congruent to 1 mod 4. The argument I just gave proves the upper bound $$ {\rm dim}(\wedge^q({\rm Sym}^2(\mathbb{R}^n))^{O(n)})\le N(q)\ . $$ Interestingly, the computations by Vit also give a computer assisted proof that, in fact, equality holds for $q\le 14$ (and $n$ large enough, i.e., in the stable case).

---

Update: In the trace of five matrices case one can prove nonvanishing as follows. The invariant $L_\lambda$ corresponding to $\lambda=(5)$ is $$ L_{\lambda}(T)=\frac{1}{5!}\sum_{\sigma\in\mathfrak{S}_5} \varepsilon(\sigma)\ {\rm tr}(A^{(\sigma(1))}\cdots A^{(\sigma(5))}) $$ when $T=A^{(1)}\wedge\cdots\wedge A^{(5)}$. Now take the following $5\times 5$ matrices. Let $A^{(1)}_{12}=A^{(1)}_{21}=1$ and all other entries zero. Let $A^{(2)}_{23}=A^{(2)}_{32}=1$ and all other entries zero. Etc. Let $A^{(5)}_{51}=A^{(5)}_{15}=1$ and all other entries zero. Namely, consider the graph given by the cycle $(12345)$. For each edge in the cycle take the adjacency matrix of the corresponding one-edge graph. If I did not mess up my computation, the corresponding specialization of $L_{\lambda}$ is equal to $1/12$.

This can be generalized for a partition $\mu$ of $q$, also with distinct parts congruent to 1 mod 4. One then defines an element $T_{\mu}=A^{(1)}\wedge\cdots\wedge A^{(q)}$ as before. Namely, build a graph $G$ with vertex set $\{1,\ldots,q\}$ made of disjoint cycles with lengths given by the parts of $\mu$. For each edge in $G$ build the adjacency matrix of the corresponding one-edge graph to get the $A$ matrices. If one has a part $\mu_a=1$ corresponding to a cycle $(j)$ then take the matrix with $A_{jj}=1$ and all other entries zero. A quick combinatorial argument shows that $L_{\lambda}(T_{\mu})$ is an invertible diagonal matrix (with rows and columns indexed by partitions). By construction products of $A$'s give zero most of the time unless $\lambda$ replicates $\mu$. This immediately implies that $N(q)$ is the exact answer to the OP's question.