Ring of invariants of some special type of subgroups of $GL_3(\mathbb C)$

Question

For $\sigma \in \mathrm{GL}_n(\mathbb C)$ and $f(x_1,...,x_n)\in \mathbb C[x_1,...,x_n]$, let $f^ \sigma (x):=f(\sigma^{-1}x)$, for $x=(x_1,...,x_n)$.

For a subgroup $G$ of $\mathrm{GL}_n(\mathbb C)$, let $\mathbb C[x_1,...,x_n]^G :=\{f\in \mathbb C[x_1,...,x_n] : f^\sigma =f ,\forall \sigma \in G\}$.

Now let $f_0,f_1,f_2 \in \mathbb C[x,y,z]$ be defined as $f_0(x,y,z)=x^3+y^3+z^3+6xyz$, $$f_1(x,y,z)=3(x^2y+y^2z+z^2x),\quad f_2(x,y,z)=3(xy^2+yz^2+zx^2).$$

Let $G=\{\sigma \in \mathrm{GL}_3(\mathbb C) : f_0^\sigma =f_0,\; (f_1^\sigma)^3=f_1^3,\; (f_2^\sigma)^3=(f_2)^3 \}$ .

Let $H=\{\sigma \in \mathrm{GL}_3(\mathbb C) : f_0^\sigma =f_0,\; f_1^\sigma=f_1,\; f_2^\sigma=f_2 \}$.

What are the orders of $G$ and $H$ ? Is $H$ a proper subgroup of $G$ ? How to show that every matrix in $G$ is unitary ? Is it true that $\mathbb C[x,y,z]^G=\mathbb C[f_0,f_1^3,f_2^3]$ ? Is it true that $\mathbb C[x,y,z,]^H=\mathbb C[f_0,f_1,f_2]$ ?

Answer 1

Note added on 26 Nov 2018: I have corrected my answer, which had a serious mistake.

For simplicity of notation, let $(x,y,z) = (x_1,x_2,x_3)$. The Hessian form associated to $f_0 = {x_1}^3+{x_2}^3+{x_3}^3+6x_1x_2x_3$ is $$ H(f_0) = \frac{\partial^2f_0}{\partial x_i\partial x_j}\,\mathrm{d}x_i\circ\mathrm{d}x_j\,. $$ The determinant of this Hessian form is easily computed to be $$ \Delta = -6^3 ({x_1}^3+{x_2}^3+{x_3}^3-3x_1x_2x_3)\, (\mathrm{d}x_1\wedge\mathrm{d}x_2\wedge\mathrm{d}x_3)^{\otimes2}. $$ Any linear transformation of $\mathbb{C}^3$ that preserves $f_0$ must preserve $\Delta$. Since $$ {x_1}^3{+}{x_2}^3{+}{x_3}^3{-}3x_1x_2x_3 = (x_1{+}x_2{+}x_3)(x_1{+}sx_2{+}s^2x_3)(x_1{+}s^2x_2{+}sx_3) $$ where $s^2+s+1=0$ (so that $s^3=1$), it follows that any linear transformation $L$ of $\mathbb{C}^3$ that preserves $f_0$ must must permute and scale the three linearly independent forms $$ \xi_i = (x_1{+}s^ix_2{+}s^{2i}x_3)\quad i = 0,1,2. $$ Thus, $L^*(\xi_i) = \lambda_i\,\xi_{\pi(i)}$ for $i=0,1,2$, where the $\lambda_i$ are nonzero and $\pi$ is a permutation of $\{0,1,2\}$. Expressing $f_0$ in terms of the $\xi_i$, we find that $f_0 = \tfrac13({\xi_0}^3{+}{\xi_1}^3{+}{\xi_2}^3)$. It now follows that $\lambda_i^3 = 1$ for $i=0,1,2$.

Conversely, for any permuation $\pi$ of $\{0,1,2\}$ and any cube roots of unity $\lambda_i$ for $i=0,1,2$, the formula $L^*(\xi_i) = \lambda_i\,\xi_{\pi(i)}$ defines a linear transformation of $\mathbb{C}^3$ that preserves $f_0$. Thus, the stabilizer of $f_0$ is a finite group of order $27\cdot 6 = 162$ (which is unitary for the Hermitian form $F=|\xi_1|^2{+}|\xi_2|^2{+}|\xi_3|^2 = 3(|x_1|^2{+}|x_2|^2{+}|x_3|^2)$). Note, though, that this shows that the first sentence of GreginGre's answer is incorrect (and renders the rest of that argument moot).

Meanwhile, we find $$ f_1 = \tfrac13({\xi_0}^3+s^2\,{\xi_1}^3+s^4\,{\xi_2}^3) \qquad\text{and}\qquad f_2 = \tfrac13({\xi_0}^3+s\,{\xi_1}^3+s^2\,{\xi_2}^3). $$ It follows that $L$, as defined above, preserves $(f_i)^3$ if and only if $\pi$ is an even permutation of $\{0,1,2\}$. Thus, $G$ has order $81$.

In order to preserve $f_1$ and $f_2$, the permutation $\pi$ must be the identity. Thus, $H$ has order $27$.

One sees that $\mathbb{C}[x,y,z]^G$ contains $f_1f_2$, so it is not equal to $\mathbb{C}[f_0,(f_1)^3,(f_2)^3]$.

On the other hand $H$ is the group that preserves each of the ${\xi_i}^3$ (since these are linear combinations of $f_0,f_1,f_2$ and vice versa) and $H$ preserves a monomial ${\xi_0}^{n_0}{\xi_1}^{n_1}{\xi_2}^{n_2}$ if and only if all of the $n_i$ are divisible by $3$. Hence $\mathbb{C}[x,y,z]^H=\mathbb{C}[{\xi_0}^3,{\xi_1}^3,{\xi_2}^3]=\mathbb{C}[f_0,f_1,f_2]$.

I think that answers all your questions.

Answer 2

It is known that the linear transformation fixing $f_0$ are exactly the transformations $(x_1,x_2,x_3)\mapsto (\lambda x_{\sigma(1)}, \mu x_{\sigma(2)}, (\lambda\mu)^{-1}x_{\sigma(3)})$, where $\sigma\in\mathfrak{S}_3$ and $\lambda^3=\mu^3=1$.

These transformations also satisfy $f_i^\sigma=\gamma_if_i$ for some $\gamma_i$ such that $\gamma_i^3=1$, for $i=1,2$. Hence $G$ consists exactly of these $54$ transformations. Notice they are all unitary.

Now for such a transformation to be in $H$, you need $\lambda^2\mu=1$ and $\lambda\mu^2=1$, that is $\lambda=\mu$, so the elements of $H$ are the transformations $(x_1,x_2,x_3)\mapsto (\lambda x_{\sigma(1)}, \lambda x_{\sigma(2)}, \lambda x_{\sigma(3)})$, where $\sigma\in\mathfrak{S}_3$ and $\lambda^3=1$, which has $18$ elements, so $H$ is a proper subgroup of $G.$ (Note that it is easy to understand the group structure of $G$ and $H$, but I'm a bit lazy now).

Anyway, a polynomial fixed by $G$ or $H$ will have to be symmetric, since $G$ and $H$ contain the symmetric group on three letters, so you cannot expect $f_1$ or $f_2$ to be fixed by $H$.

It seems that $\mathbb{C}[x_1,x_2,x_3]^H=\mathbb{C}[\sigma_1^3,\sigma_2^3,\sigma_3,\sigma_1\sigma_2]$ and $\mathbb{C}[x_1,x_2,x_3]^G=\mathbb{C}[\sigma_1^3,\sigma_2^3,\sigma_3]$, where $\sigma_1,\sigma_2,\sigma_3$ are the elementary symmetric polynomials, but I may be wrong, since I'm not an expert in invariant theory.