# Subgroups of $\mathrm{SO}(A_0, \mathbb{F}_p)$

Let $$n \geq 3$$. Let $$A_0$$ denote the $$n \times n$$ symmetric matrix with $$1$$'s on the antidiagonal and $$0$$'s everywhere else. We can define the associated special orthogonal group $$\mathrm{SO}(A_0, \mathbb{F}_p) := \{ g \in \mathrm{SL}(n, \mathbb{F}_p) : g^t A_0 g = A_0 \}.$$ We can suppose that $$p \neq 2$$.

This group appears in The average size of the $$2$$-Selmer group of Jacobians of hyperelliptic curves... by Bhargava and Gross (arxiv link) and several follow-up works. This group acts on the group of $$n \times n$$ symmetric matrices by conjugation, and Bhargava and Gross study sizes of certain orbits under this action.

I'm wondering what is known about the general structure of subgroups of $$\mathrm{SO}(A_0, \mathbb{F}_p)$$? In particular, do we understand the subgroups of small index?

Morally, subgroups of small index could potentially be large stabilizer groups for the action, corresponding to small orbit sizes. I'm actually looking for methods to show that there aren't too many small orbits "except for expected ones", and I'm wondering if abstract group theory might be enough.

As a closely related tiny question, I note that I think that of the subgroup structure of $$\mathrm{SO}(A_0, \mathbb{F}_p)$$ as being essentially the same as the subgroup structure of $$\mathrm{SO}(\mathbb{F}_p)$$ --- is that right?

• I think this post gives the answer, more or less. mathoverflow.net/q/175933/74819. Basically, Witt extension theorem shows that the orbits must be large. About your tiny question, I assume that by SO(Fp) you mean the standard inner product. Note that if p=1 mod 4, then they are the same, as the form is completely split. Commented Nov 17, 2021 at 3:03
• Also, Larsen and Pink have a general result about algebraic groups that van be applied here, see their paper "Finite subgroups of algebraic groups". Commented Nov 17, 2021 at 3:27

A useful invariant of quadratic forms is the Witt ring $$W$$. The 1-dimensional form $$x^2$$ correspond to $$1\in W$$. The standard form, which I will denote $$A_1$$, satisfies $$[A_1]=n\cdot 1 \in W$$. It is easy to see that $$[A_0]\in W$$ is 0 if $$n$$ is even and it 1 if $$n$$ is odd. The order of 1 in $$W$$ is 2 if $$p$$ is 1 mod 4 and it is 4 if $$p$$ is 3 mod 4. It follows that $$A_0$$ an d$$A_1$$ are equivalent if $$p$$ is 1 mod 4 or if $$p$$ is 3 mod 4 and $$n$$ is 0 or 1 mod 4. In case both $$p$$ and $$n$$ are 3 mod 4, $$A_0$$ and $$A_1$$ are not equivalent, but differ (up to equivalence) by a (non-square) scalar multiplication. In all of these cases $$\text{SO}(A_0,\mathbb{F}_p)$$ and $$\text{SO}(\mathbb{F}_p)$$ are conjugated in $$\text{GL}(\mathbb{F}_p)$$. In the remining case, where $$p$$ is 3 mod 4 and $$n$$ is 2 mod 4, these groups are non-isomorphic and their "subgroup structure" is different. For example, the split group $$\text{SO}(A_0,\mathbb{F}_p)$$ contains a copy of $$(F_p^*)^{n/2}$$, while $$\text{SO}(\mathbb{F}_p)$$ does not.
I am quite confident that a subgroup of minimal index in either group is the stabilizer $$Q$$ of an isotropic line in $$\mathbb{F}_p^n$$. It is certainly minimal among algebraic subgroups. The orbit of an isotropic line in the projective space $$\mathbb{P}^{n-1}(\mathbb{F}_p)$$ is the zero locus of the corresponding quadratic form, thus it is a variety of dimension $$n-2$$. It follows that $$[G:Q]$$ is $$\sim p^{n-2}$$.
$$Q$$ is a one of the maximal parabolic subgroups - these are the stabilizers of isotropic subspaces of $$\mathbb{F}_p^n$$ (of dimension $$1,\dots,[n/2]$$ for the split form $$A_0$$). The others are also of relatively small index, but bigger then that of $$Q$$. Other subgroups of small index are reductive subgroups, such as the stabilizer of an anisotropic vector in $$\mathbb{F}_p^n$$, which is of codimension $$n-1$$, thus of index $$\sim p^{n-1}$$. Again, you can take stabilizers of subspace of higher dimension, but you will pay for this with a bigger index.
• In any dimension $n\ge2$, there are two equivalence classes of nonsingular quadratic forms on $(F_p)^n$. But when $n$ is odd, there are two inequivalent quadratic forms which are scalar multiples of each other. Hence the two corresponding orthogonal groups are isomorphic for $n$ odd. Commented Nov 19, 2021 at 23:12