What are the possible automorphism groups of a principally polarized abelian variety $(A,\lambda)$ of dimension $g,$ say an abelian surface ($g=2$) over the complex numbers or algebraic closure of a finite field? The fact that the moduli stack $A_g$ is of finite diagonal (over the integers) implies that the automorphism groups are all finite, but do we know more? Like the size.

When $g=1$ this is given in Silverman I, p.103.

**Edit:** Let me make the question more specific. Let $(A,\lambda)$ be an $\mathbb F_q$-point of $A_g$ (i.e. an abelian variety $A$ over $\mathbb F_q$ of dimension $g$ and a principal polarization $\lambda$). We want to consider its automorphism group (over $\mathbb F_q$).

Let $\pi:A_{g,N}\to A_g$ be the natural projection, where $A_{g,N}$ is the moduli stack of p.p.a.v. of dimension $g$ with a level $N$ structure (a symplectic isomorphism $H^1(A,Z/N)\to(Z/N)^{2g}$). We always assume $q$ is prime to $N.$ Note that $\pi$ is a $G$-torsor, for $G=GSp(2g,Z/N),$ so it gives a surjective homomorphism $\pi_1(A_g)\to G.$ The sheaf $\pi_*\mathbb Q_l$ on $A_g$ is lisse (even locally constant), corresponding to the representation of $\pi_1(A_g)$ obtained from the regular representation $\mathbb Q_l[G]$ of $G$ and the projection $\pi_1(A_g)\to G.$ For any $\mathbb F_q$-point $x$ of $A_g,$ the local trace $\text{Tr}(Frob_x,(\pi_*\mathbb Q_l)_{\overline{x}})$ is either $|G|$ or 0, depending on $Frob_x\in\pi_1(A_g)$ is mapped to 1 in $G$ or not.

We have isomorphisms $H^i_c(A_{g,N},\mathbb Q_l)=H^i_c(A_g,\pi_*\mathbb Q_l).$ By Lefschetz trace formula, applied to both $\mathbb Q_l$ on $A_{g,N}$ and $\pi_*\mathbb Q_l$ on $A_g,$ we have

$$|A_{g,N}(\mathbb F_q)|=|G|\sum_{x\in S} 1/\#Aut(A_x,\lambda_x),$$

where $S$ is the subset of $[A_g(\mathbb F_q)]$ consisting of points $x$ such that all $N$-torsion points of the abelian variety $A_x$ are rational over $\mathbb F_q$ (i.e. $|A_x[N](\mathbb F_q)|=N^{2g}$), and $(A_x,\lambda_x)$ is the pair corresponding to $x.$ This equation gives some constraints (one for each $N$) that $|Aut(A,\lambda)|$ must satisfy. In particular, when $g=N=2$ and $q=3,$ we have $|A_{2,2}(\mathbb F_3)|=10$ and $|G|=720$ (in this case $G$ is the symmetric group $S_6$), and this becomes a puzzle of solving $$ 1/72 = \sum 1/n_i, $$ and the $n_i$'s satisfy some additional conditions. Any idea on how to solve it? I'm considering the contributions of the two parts in $A_2,$ one for Jacobians of smooth genus 2 curves and one for Jacobians of stable singular ones $E_1\times E_2$. Any suggestion is appreciated.

**Edit**: Maybe it's easier to solve it over $\mathbb F_5,$ since the (orders of the) automorphism groups of smooth genus 2 curves over finite fields of characteristic 5 is known.

Complex Abelian Varieties. Their Chapter 13 is about this topic; e.g. they give rather precise information in the case of abelian surfaces in Section 13.4. – Lennart Meier Apr 23 '13 at 20:10