What are the automorphism groups of (principally polarized) abelian varieties?

Question

What are the possible automorphism groups of a principally polarized abelian variety $(A,\lambda)$ of dimension $g,$ say an abelian surface ($g=2$) over the complex numbers or algebraic closure of a finite field? The fact that the moduli stack $A_g$ is of finite diagonal (over the integers) implies that the automorphism groups are all finite, but do we know more? Like the size.

When $g=1$ this is given in Silverman I, p.103.

Edit: Let me make the question more specific. Let $(A,\lambda)$ be an $\mathbb F_q$-point of $A_g$ (i.e. an abelian variety $A$ over $\mathbb F_q$ of dimension $g$ and a principal polarization $\lambda$). We want to consider its automorphism group (over $\mathbb F_q$).

Let $\pi:A_{g,N}\to A_g$ be the natural projection, where $A_{g,N}$ is the moduli stack of p.p.a.v. of dimension $g$ with a level $N$ structure (a symplectic isomorphism $H^1(A,Z/N)\to(Z/N)^{2g}$). We always assume $q$ is prime to $N.$ Note that $\pi$ is a $G$-torsor, for $G=GSp(2g,Z/N),$ so it gives a surjective homomorphism $\pi_1(A_g)\to G.$ The sheaf $\pi_*\mathbb Q_l$ on $A_g$ is lisse (even locally constant), corresponding to the representation of $\pi_1(A_g)$ obtained from the regular representation $\mathbb Q_l[G]$ of $G$ and the projection $\pi_1(A_g)\to G.$ For any $\mathbb F_q$-point $x$ of $A_g,$ the local trace $\text{Tr}(Frob_x,(\pi_*\mathbb Q_l)_{\overline{x}})$ is either $|G|$ or 0, depending on $Frob_x\in\pi_1(A_g)$ is mapped to 1 in $G$ or not.

We have isomorphisms $H^i_c(A_{g,N},\mathbb Q_l)=H^i_c(A_g,\pi_*\mathbb Q_l).$ By Lefschetz trace formula, applied to both $\mathbb Q_l$ on $A_{g,N}$ and $\pi_*\mathbb Q_l$ on $A_g,$ we have

$$|A_{g,N}(\mathbb F_q)|=|G|\sum_{x\in S} 1/\#Aut(A_x,\lambda_x),$$

where $S$ is the subset of $[A_g(\mathbb F_q)]$ consisting of points $x$ such that all $N$-torsion points of the abelian variety $A_x$ are rational over $\mathbb F_q$ (i.e. $|A_x[N](\mathbb F_q)|=N^{2g}$), and $(A_x,\lambda_x)$ is the pair corresponding to $x.$ This equation gives some constraints (one for each $N$) that $|Aut(A,\lambda)|$ must satisfy. In particular, when $g=N=2$ and $q=3,$ we have $|A_{2,2}(\mathbb F_3)|=10$ and $|G|=720$ (in this case $G$ is the symmetric group $S_6$), and this becomes a puzzle of solving $$ 1/72 = \sum 1/n_i, $$ and the $n_i$'s satisfy some additional conditions. Any idea on how to solve it? I'm considering the contributions of the two parts in $A_2,$ one for Jacobians of smooth genus 2 curves and one for Jacobians of stable singular ones $E_1\times E_2$. Any suggestion is appreciated.

Edit: Maybe it's easier to solve it over $\mathbb F_5,$ since the (orders of the) automorphism groups of smooth genus 2 curves over finite fields of characteristic 5 is known.

Answer 1

The standard proof of finiteness goes as follows: the polarization defines a positive involution * on the endomorphism algebra, and so the automorphisms are the elements of End(A) tensor the reals that are in End(A) and satisfy a*a=1. Thus the set of automorphisms is the intersection of a discrete set and a compact set. Let phi(n) be the degree of the field you get by adjoining an nth root of 1 to Q. Then certainly there exist abelian varieties of dimension g on which the nth roots of 1 act if phi(n) divides 2g (for any CM-field E there is an abelian variety with complex multiplication by E). However, unlike the elliptic curve case, there are other possibilities.

Answer 2

If we start with the same problem over the complex numbers, then the automorphism groups of principally polarised abelian varieties of dimension $g$ are exactly the stabilisers of $\mathrm{Sp}_{2g}(\mathbb Z)$ in its action on the Siegel upper half space $\mathbb H_g$. These are finite groups and it is a general fact that a finite subgroup of $\mathrm{Sp}_{2g}(\mathbb Z)$ fixes a point of $\mathbb H_g$. Hence the answer in that case are that the automorphism groups are exactly the maximal finite subgroups of $\mathrm{Sp}_{2g}(\mathbb Z)$. Note that just as for $\mathrm{SL}_g(\mathbb Z)$ (which is a subgroup of $\mathrm{Sp}_{2g}(\mathbb Z)$) the classification of such finite subgroups quickly becomes untractable (for increasing $g$).

If we try to use this to get lower bounds for the case when the base field is the closure of $\mathbb Z/p$ we can start with a pair $(A,G)$ of a p.p.a.v. and a finite group $G$ of automorphisms over a field of characteristic. We can then (after appropriately changing fields) assume that the base field is the fraction field of a DVR $R$ of mixed characteristic with finite residue field of characteristic $p$ and that $A$ has semi-stable reduction over $R$. The action of $G$ extends and we then get $G$ as a subgroup of of the automorphism group of a semi-abelian variety with a principal polarisation of the abelian part of characteristic $p$. Note that if $G$ is large (for some definition of "large") then the reduction is necessarily an abelian variety as the automorphism of a properly semi-abelian variety is "smaller". This should give a lot of characteristic $p$ examples.

This however will not get everything as there are (already in the case of $g=1$) pairs $(A,G)$ that don't lift to characteristic $0$. It does however give all such pairs for which the order of $G$ is prime to $p$. In principle it should be possible to resolve the problem with the approach suggested in Milne's answer. However, I think there is a problem (apart from the fact that the algebraic problem quickly becomes intractable) in that it is a somewhat tricky problem to figure out which pairs consisting of a Rosati involution and a stable order in the isogeny algebra are realisable by principal polarisations. Still there are some constructions that can be used: One can take the product of two principally polarised automorphism group. One can also tensor a principally polarised AV with a positive definite unimodular hermitian form over the endomorphism ring with the Rosati involution. The latter includes starting with a positive definite integral unimodular form of rank $g$ and realising its automorphism group as the automorphism group of the product of $g$ copies of any elliptic curve. Starting instead with a supersingular elliptic curve probably give larger examples.

The problem for finite fields instead of the algebraic closure of one is even more unpredictable.

Answer 3

If you fix the 2-torsion points, then all you have is the x->-x involution; so Z/2 times SP_2g(2) is a bound from above. On the other hand, if you take A to be g-fold "power" of some elliptic curve, I think you get rather close to this bound.

Answer 4

Grushevsky's paper GEOMETRY OF Ag AND ITS COMPACTIFICATIONS, remark 2.9 sketches why the diagonal of A_g is finite; maybe you can make that proof effective?