# Is there an invariant theory explanation of the orbit structure of GL₂ acting on second-diagonal symmetric matrices by g∙X = gXJg^tJ ?

## Statement of the Specific Result

Let $J$ denote the matrices with ones on the "second diagonal", meaning the diagonal between the (1,n) and (n,1) entry, and zeros elsewhere. So in the case $n=2$, which is all we'll be concerned with here:

$$J=\begin{pmatrix} 0&1 \newline 1&0 \end{pmatrix}$$

Let the transpose of $g$ be denoted by ${}^tg$.

Let $F$ be a field (not necessarily alg. closed: for example, a number field), and consider the action of $G:=\mathrm{GL}_2(F)$ on the vector space $\mathcal{L}$ of matrices with entries in $F$ which are symmetric about the second diagonal, by $g\cdot X= g X J{}^tgJ$ for $g\in\mathrm{GL}_2(F)$. In explicit coordinates,

$$g=\begin{pmatrix}a&b \newline c&d\end{pmatrix}$$

$$X = \begin{pmatrix}\gamma&\beta \newline \alpha&\gamma\end{pmatrix}$$

$$J{}^tgJ=\begin{pmatrix}d&b \newline c&a\end{pmatrix}$$

so that this procedure of "taking the transpose of $g$ and conjugating by $J$" amounts to taking the transpose of $g$ along the second diagonal. (See below for Context). In several papers, I find the following stated:

1. $G$ acts on $\mathcal{L}$ with an open orbit.
2. Given a representative $X_0\in\mathcal{L}$ of the open orbit, the stabilizer $G_0(X_0)$ is in general reductive, and in this specific example, a one-dimensional torus.
3. An point $X$ is called generic if the stabilizer $G_0(X)$ is of type $G_0(X_0)$, and an orbit is called generic if one, equiv. all, its points are generic points. A complete set of representatives of the generic orbits (with exactly one representative from each orbit) is given by the matrices $$\begin{pmatrix} 0&\beta\\ \alpha&0 \end{pmatrix}$$ with $$\alpha,\beta\in {F^*}^2\backslash F^*$$ (so with the diagonal element $\gamma=0$ and the off-diagonal elements ranging over nonzero square classes of $F^*$ independently).

What I would like to know...

1. is there any tidy way of "characterizing" generic points or orbits, as referred to in items 1 and 2?
2. Is there any conceptual or "coordinate-free" way of characterizing the representative set given in item 3?

I have a feeling that "standard, classical" invariant theory, especially that of the symplectic group, may give an answer to this. I am not sufficiently familiar with the invariant-theory literature to find this, so if you could point me to a specific reference that I could read and which would allow me to answer these questions, that would be great. Although I have a (partial) confirmation of these facts by brute-force matrix calculations, these are not really ideal to use for my purposes, nor is it clear that they could be carried out by anyone in higher dimensions than 2!

## The context, and more on why I expect invariant theory to play a role

The context of this problem is that the symplectic group $\mathrm{Sp}_4(F)$ has a standard ("Siegel") parabolic $P$ with Levi factor $M$ isomorphic to $G$, which embeds into $\mathrm{Sp}_4$ by $\mathrm{diag}(g,J{}^t g^{-1}J)$ (according to one of the two or so common matrix models of $\mathrm{Sp}_4$). The nilpotent radical of $P$ can be identified with $\mathcal{L}$, and the conjugation action of $M$ is then identified with the action above.

If one considers the analogous situation with $\mathrm{SO}_{5}$ (say), and the parabolic $Q$ with Levi factor $\mathrm{GL_1}\times \mathrm{SO}_{3}$, the nilpotent radical of $Q$ can be identified with "row vectors of length 3", and the genericity condition can clearly be expressed as a row-vector representative having non-zero length. Then it is easy to divide "generic" vectors into different $M(F)$-conjugacy classes by the square-class of their (non-zero) lengths. So this very simple invariant-theory interpretation gives me the feeling there is something conceptual going on in the situation I have described, which I am unfortunately missing at the moment.

Thanks for reading and I will greatly appreciate any help!

• Brilliant user name! Sep 20, 2010 at 2:42

I don't completely understand every part of your question but I will get the ball rolling and edit if necessary. The short answer to your question is yes (at least in the case of infinite fields which is all I know about).

The orbits can be characterised in a coordinate independent way by an orbit-type stratifaction of the orbit space. The orbit space is diffeomorphic to the semialgebraic variety defined on the space of invariant polynomial generators with inequalites given by the Procesi and Schwarz type approach. This space admits a primary stratification (in the sense of Whitney) and the strata in the two diffeomorphic spaces correspond.

In your case I seem to get that (for $F = \mathbb{C}$), the ring of invariant polynomials are generated by the real and imaginary parts of

$\gamma^2 +\alpha\beta$

and

$2|\gamma|^2 + |\alpha|^2 + |\beta|^2$

If you think of these as functions on $\mathbb{C}^3$ or $\mathbb{R}^6$ you can take gradients and then dot products to get a Grammian matrix. Then the equations defined by setting the various minors of the matrix equal to zero will identify the boundary of a semialgebraic region of $\mathbb{R}^3$. On the various pieces of the boundary of different dimensions the orbit type (equivalence class defined by conjugations of stabiliser groups) will be different.

The fact that the orbits can be characterised by a matrix of the form you give is then related to the fact that you can solve

$|\alpha|^2 + |\beta|^2 = C_1$

$\alpha\beta= C_2 + i C_3$

for arbitrary (C_1,C_2,C_3) in the semialgebraic variety.

If this is in anyway helpful let me know if I can add more details.

• Thanks QQJ, but I am afraid my inexperience in invariant theory keeps me from understanding your notation. It seems to me now that if you ask the same question in an alternate model of the symplectic group (where the form is defined by antidiag(I,-I) instead of antidiag(J,-J)) then the problem essentially reduces to the classifying the symmetric forms over the field, which is a very classical and fully solved problem. What are called the "generic" orbits are the orbits consisting of non-degenerate forms, in the usual sense. I will try to write this up this afternoon and post here. Sep 4, 2010 at 18:38

I have been able to determine that this is really, if not exactly, then almost exactly, the question of finding the equivalence classes of bilinear forms over $F$, although with everything conjugated in order to make this non-obvious at first glance. The process of stating the question really helped clarify what was going on here, so thanks to everyone for taking a look, and I will try to come up with something a bit more interesting next time I post! Also thanks to QQJ for your answer; I am afraid I am not really expert enough in the fields you allude to to determine whether it answers the original question, but there are undoubtedly some worthwhile ideas there.

• Yes, since $J^2=I,$ your formula for the action can be rewritten as $g\cdot Y=gY{{\,}^{t}\!g},$ where $Y=XJ$ is a symmetric matrix, $Y=\,^{t}\!Y.$ By the way, if you insist on writing the transpose on the left, make sure to insert small space after the previous symbol, because an expression like $J^{t} X$ appears to be $J$ transpose times $X,$ not $J$ times $X$ transpose. Also, in this context, the letter $J$ is a piece of standard notation with a different meaning, viz the skew-symmetric matrix of the symplectic form. Sep 6, 2010 at 1:32