# Invariant ring of $\textrm{Sym}^2(\wedge^2\mathbb{R}^4)$ under $\textrm{SO}(4)$

Consider the representation of $$\textrm{SO}(4)$$ on $$\textrm{Sym}^2(\wedge^2\mathbb{R}^4)$$ induced by the standard representation of $$\textrm{SO}(4)$$ on $$\mathbb{R}^4$$. I am interested in the ring of invariants of this representation, i.e. the ring of all polynomial functions on $$\textrm{Sym}^2(\wedge^2\mathbb{R}^4)$$ that are invariant.

Here is a way of constructing some of these invariants. The space of $$\textrm{SO}(4)$$-linear maps $$\wedge^2\mathbb{R}^4\to\wedge^2\mathbb{R}^4$$ is $$2$$-dimensional: We have the identity $$I$$ and the Hodge star operator $$\star$$. Thus the coefficients of the polynomial $$\det(x\cdot I+y\cdot \star+A)\in\mathbb{R}[x,y]$$are polynomials in the entries of $$A\in\textrm{Sym}^2(\wedge^2\mathbb{R}^4)$$ which are $$\textrm{SO}(4)$$-invariant. My question is: Do these polynomials already generate the ring of invariants?

The answer is 'no', though I don't know an easy way to see this without doing an explicit calculation. Here is where to look though, if you want to do the calculation yourself:

Things work out a bit better if one uses indeterminates $$z = x+y$$ and $$w = x-y$$. Then one has an expansion $$\det\bigl(x{\cdot}I + y{\cdot}\star + A\bigr) = \sum_{0\le i,j\le 3} P_{ij}(A)\,z^{3-i}w^{3-j}$$ where $$P_{ij}(A)$$ is a polynomial in $$A$$ of degree $$i{+}j$$.

One finds that $$P_{00}=1$$. Meanwhile, the polynomials $$P_{10}$$, $$P_{01}$$, $$P_{20}$$, $$P_{11}$$, $$P_{02}$$, $$P_{30}$$, $$P_{21}$$, $$P_{12}$$, and $$P_{03}$$ are independent and generate all of the polynomial invariants of degree $$3$$ or less. Moreover, the three quartic polynomials $$P_{3,1}$$, $$P_{2,2}$$, and $$P_{1,3}$$ are algebraically independent over the ring generated by the lower degree $$P_{ij}$$.

However, the $$P_{ij}$$ with $$i+j\le 4$$ do not generate all of the invariant polynomials of degree $$4$$. There is one further quartic polynomial invariant, say $$Q$$, that is not a polynomial in the $$P_{ij}$$ with $$i+j\le 4$$. Since the remaining three coefficients $$P_{3,2}$$, $$P_{2,3}$$ and $$P_{3,3}$$ are all of degree 5 or higher, $$Q$$ does not lie in the ring generated by the $$P_{ij}$$.

Added detail: $$\mathrm{SO}(4)$$ preserves the splitting $$\Lambda^2(\mathbb{R}^4)= \Lambda^2_+(\mathbb{R}^4)\oplus\Lambda^2_-(\mathbb{R}^4)$$ into self-dual and anti-selfdual forms, i.e., the split into the eigenspaces of $$\star$$, which has trace-zero and square $$\star^2 = I$$. It follows that we can write block decompositions $$\star = \begin{pmatrix} I_3&0_3\\0_3&-I_3\end{pmatrix} \quad\text{and}\quad A = \begin{pmatrix} a&b\\b^T&c\end{pmatrix},$$ where $$a=a^T$$, $$b$$ and $$c=c^T$$ are $$3$$-by-$$3$$ matrices. The action of $$\mathrm{SO}(4)$$ on $$A$$ can be described as $$g\cdot A = \begin{pmatrix} g_+ag_+^T&g_+bg_-^T\\g_-b^Tg_+^T&g_-cg_-^T\end{pmatrix}$$ where $$g_\pm:\mathrm{SO}(4)\to\mathrm{SO}(3)$$ are homomorphisms such that $$(g_+,g_-):\mathrm{SO}(4)\to\mathrm{SO}(3)\times\mathrm{SO}(3)$$ is surjective, with kernel $$\{\pm I_4\}$$. We can write $$\det(x{\cdot}I_6 + y{\cdot}\star + A) = \det\begin{pmatrix} zI_3+a & b\\ b^T& wI_3 + c\end{pmatrix} = \sum_{0\le i,j\le 3} P_{ij}(A)\,z^{3-i}w^{3-j},$$ and we can refine our notation by writing $$P_{ij}(A) = P_{ij}(a,b,c)$$.

Now, because of the way $$a$$, $$b$$, and $$c$$ transform under the action of $$\mathrm{SO}(4)$$, we see, for example, that $$\mathrm{tr}(a^k)$$ and $$\mathrm{tr}(c^l)$$ are invariant polynomials, as is $$\mathrm{tr}(bb^T)$$ (though $$\mathrm{tr}(b)$$ is not), and, indeed, we see, by inspection, that, for example, $$P_{10} = \mathrm{tr}(a)\quad\text{and}\quad P_{01} = \mathrm{tr}(c),$$ and a little reflection shows that $$P_{20} = \tfrac12\bigl(\mathrm{tr}(a)^2-\mathrm{tr}(a^2)\bigr) \quad\text{and}\quad P_{02} = \tfrac12\bigl(\mathrm{tr}(c)^2-\mathrm{tr}(c^2)\bigr)$$ while it's not hard to see that $$P_{11} = \mathrm{tr}(a)\mathrm{tr}(c) - \mathrm{tr}(bb^T).$$ To go further, imagine that we have an alphabet of 4 letters, $$a$$, $$b$$, $$b^T$$, and $$c$$, and we declare a valid `word' to be string of these letters subject to the following rules: $$a$$ can be followed only by $$a$$ or $$b$$, $$b$$ can only be followed by $$c$$ or $$b^T$$, $$c$$ can only be followed by $$c$$ or $$b^T$$, $$b^T$$ can only be followed by $$a$$ or $$b$$, and finally, that the number of $$b$$s in the word is the same as the number of $$b^T$$s. Thus, for example, $$a^4$$, $$abb^T$$, and $$abcb^T$$ are valid words, but $$abc$$ is not valid. What one sees that is a vaild word $$w$$, when interpreted as a product of matrices, transforms under $$\mathrm{SO}(4)$$ into $$g_\pm w g_\pm^T$$, and hence $$\mathrm{tr}(w)$$ is an invariant polynomial for any valid word.

Now we can go on: For example $$P_{30} = \det(a)$$ which is a weighted homogeneous polynomial in $$\mathrm{tr}(a)$$, $$\mathrm{tr}(a^2)$$, and $$\mathrm{tr}(a^3)$$, and similarly with $$P_{03}= \det(c)$$. Moreover, one finds that $$P_{21} = P_{20}P_{01}-P_{10}(P_{10}P_{01}-P_{11}) +\mathrm{tr}(abb^T)$$ with a similar formula for $$P_{12}$$. Thus, all of the polynomials $$P_{ij}$$ for $$0 are expressed as polynomials in the 9 (independent) invariants $$\mathrm{tr}(a),\mathrm{tr}(c),\ \mathrm{tr}(a^2),\mathrm{tr}(c^2),\mathrm{tr}(bb^T),\ \mathrm{tr}(a^3),\mathrm{tr}(abb^T),\mathrm{tr}(cb^Tb), \mathrm{tr}(c^3),$$ and conversely. These are all of the invariants of degree $$3$$ or less. (Of course, $$\mathrm{tr}(a^4)$$ is a polynomial in these. Not all valid words yield an invariant that cannot be expressed as a polynomial in invariants of lower degree.)

But now, one sees where the discrepancy comes in. The quartic polynomials $$P_{31}$$, $$P_{22}$$, $$P_{13}$$ can be expressed in terms of the 'word-trace' invariants, but there are ony three of them, whereas there are four new quartic word-trace invariants: $$\mathrm{tr}(aabb^T),\ \mathrm{tr}(bb^Tbb^T),\ \mathrm{tr}(ccb^Tb),\ \mathrm{tr}(abcb^T)$$ and it is not hard to see that no nontrivial linear combination of them can be written as a polynomial in the word-trace invariants of lower degree.

In fact, the situation gets worse: There are only two $$P_{ij}$$ of degree 5, but there are four new word-trace invariants of degree 5, and $$P_{33}$$ is the only one of degree 6 while there are two new word-trace invariants of degree 6.