Subgroups of $\mathrm{SO}(A_0, \mathbb{F}_p)$

Question

Let $n \geq 3$. Let $A_0$ denote the $n \times n$ symmetric matrix with $1$'s on the antidiagonal and $0$'s everywhere else. We can define the associated special orthogonal group $$ \mathrm{SO}(A_0, \mathbb{F}_p) := \{ g \in \mathrm{SL}(n, \mathbb{F}_p) : g^t A_0 g = A_0 \}. $$ We can suppose that $p \neq 2$.

This group appears in The average size of the $2$-Selmer group of Jacobians of hyperelliptic curves... by Bhargava and Gross (arxiv link) and several follow-up works. This group acts on the group of $n \times n$ symmetric matrices by conjugation, and Bhargava and Gross study sizes of certain orbits under this action.

I'm wondering what is known about the general structure of subgroups of $\mathrm{SO}(A_0, \mathbb{F}_p)$? In particular, do we understand the subgroups of small index?

Morally, subgroups of small index could potentially be large stabilizer groups for the action, corresponding to small orbit sizes. I'm actually looking for methods to show that there aren't too many small orbits "except for expected ones", and I'm wondering if abstract group theory might be enough.

As a closely related tiny question, I note that I think that of the subgroup structure of $\mathrm{SO}(A_0, \mathbb{F}_p)$ as being essentially the same as the subgroup structure of $\mathrm{SO}(\mathbb{F}_p)$ --- is that right?

Answer 1

(Edit made in view of a comment by Richard Lyons correcting an inaccuracy in my former answer)

---

A useful invariant of quadratic forms is the Witt ring $W$. The 1-dimensional form $x^2$ correspond to $1\in W$. The standard form, which I will denote $A_1$, satisfies $[A_1]=n\cdot 1 \in W$. It is easy to see that $[A_0]\in W$ is 0 if $n$ is even and it 1 if $n$ is odd. The order of 1 in $W$ is 2 if $p$ is 1 mod 4 and it is 4 if $p$ is 3 mod 4. It follows that $A_0$ an d$A_1$ are equivalent if $p$ is 1 mod 4 or if $p$ is 3 mod 4 and $n$ is 0 or 1 mod 4. In case both $p$ and $n$ are 3 mod 4, $A_0$ and $A_1$ are not equivalent, but differ (up to equivalence) by a (non-square) scalar multiplication. In all of these cases $\text{SO}(A_0,\mathbb{F}_p)$ and $\text{SO}(\mathbb{F}_p)$ are conjugated in $\text{GL}(\mathbb{F}_p)$. In the remining case, where $p$ is 3 mod 4 and $n$ is 2 mod 4, these groups are non-isomorphic and their "subgroup structure" is different. For example, the split group $\text{SO}(A_0,\mathbb{F}_p)$ contains a copy of $(F_p^*)^{n/2}$, while $\text{SO}(\mathbb{F}_p)$ does not.

I am quite confident that a subgroup of minimal index in either group is the stabilizer $Q$ of an isotropic line in $\mathbb{F}_p^n$. It is certainly minimal among algebraic subgroups. The orbit of an isotropic line in the projective space $\mathbb{P}^{n-1}(\mathbb{F}_p)$ is the zero locus of the corresponding quadratic form, thus it is a variety of dimension $n-2$. It follows that $[G:Q]$ is $\sim p^{n-2}$.

$Q$ is a one of the maximal parabolic subgroups - these are the stabilizers of isotropic subspaces of $\mathbb{F}_p^n$ (of dimension $1,\dots,[n/2]$ for the split form $A_0$). The others are also of relatively small index, but bigger then that of $Q$. Other subgroups of small index are reductive subgroups, such as the stabilizer of an anisotropic vector in $\mathbb{F}_p^n$, which is of codimension $n-1$, thus of index $\sim p^{n-1}$. Again, you can take stabilizers of subspace of higher dimension, but you will pay for this with a bigger index.