## The automorphism group of a hyperelliptic curve

Let $C$ be the projective smooth genus 2 curve defined by $y^2=x^5-x$ over $\mathbb F_5.$ What is the order of its automorphism group (automorphisms over $\mathbb F_5$)?

I have seen different answers. In Hartshorne's Algebraic Geometry, p. 306, the answer is $2p(p^2-1)=240.$ In INFORMATION Volume 8, Number 6, pp. 837-844, Isomorphism classes of genus-2 hyperelliptic curves over finite fields $\mathbb F_{5^m},$ by L. Hernández Encinas and J. Muñoz Masqué, theorem 2, the answer is $|A_{4221}|=20$ (using notations there). A professor (let me not to mention the name for now) told me the order is 120.

Maybe I misunderstand some of the references above?

-
 I re-tagged it, adding algebraic-curves and automorphism-groups. Hope it's ok. – Tim Dokchitser Nov 25 2010 at 21:12

According to Magma it is 120, and it is an extension of $A_5$ by $C_2$ (A:=AutomorphismGroup(HyperellipticCurve(Polynomial(GF(5),[0,-1,0,0,0,1])))), and over $F_{25}$ or over $\bar F_5$ it is 240.

Edit: Hartshorne works over an algebraically closed field, so Exc. 2.5 on p.305 proves that over $\bar F_5$ the automorphism group has order 240. Explicitly, it is generated by

$\alpha: x\mapsto x+1, y\mapsto y$ of order 5,

$\beta: x\mapsto \frac{1}{x+1}, y\mapsto \frac{y}{(1+x)^3}$ of order 6,

$\gamma: x\mapsto 2x, y\mapsto \sqrt{2}y$ of order 8.

Actually, it is clear that the group they generate has order 240 and not less, because $\beta^3$ is not the hyperelliptic involution and $\gamma^4$ is. On the other hand, as Dan explains, you cannot get more than a double cover of $PGL(2,F_5)$, so this is the whole group. Over $F_5$ however, the automorphism group is generated by $\alpha, \beta$ and $\gamma^2$, and it has order 120.

-
 So the involution is defined over F_25? – shenghao Nov 25 2010 at 18:19 The involution is definitely defined over $F_5$... Let me have another look at this... – Tim Dokchitser Nov 25 2010 at 18:29

The following is wrong:

The correct answer is 240. The unnamed professor was probably thinking about the reduced automorphism group (NB. The reduced automorphism group of a hyperelliptic curve is the quotient of the automorphism group by the normal subgroup generated by the hyperelliptic involution.)

It is quite easy to describe these explicitly as well. Your curve is the unique double cover of $\mathbb P^1_{\mathbb F_5}$ which is branched over all six points which are rational over $\mathbb F_5$. The reduced automorphism group always consists of the automorphisms which preserve the unordered set of branch points, so in this case it is $PGL(2,\mathbb F_5)$. This group has order $120$.

I think I've got it now. There is a short exact sequence $1 \to \mu_2 \to \mathrm{Aut} C \to PGL(2,\mathbb F_5) \to 1$, where $\mathrm{Aut} C$ denotes the group of automorphisms defined over $\mathbb F_{25}$ or equivalently $\overline{\mathbb F}_5$.

Here is what I think is the most natural description of $\mathrm{Aut} C$ and the maps above. Let $G$ be the product $GL(2,\mathbb F_5) \times \overline{\mathbb F}_5^\ast$. Let $\Delta$ denote the normal subgroup $\{(z\cdot \mathrm{id},z^{3}) | z \in \mathbb F_5^\ast \}$ of $G$. We can let $G$ act on $x$ and $y$ via $(\gamma,\rho) \ast (x,y) \mapsto (\frac{ax+b}{cx+d},\frac{\rho y}{(cx+d)^3})$. Then $\Delta$ acts trivially, so we get an action of $G/\Delta$.

An explicit computation shows that an element $(\gamma,\rho)$ of $G$ preserves the curve $y^2 = x^5 + x$ if and only if $\det\gamma =\rho^2$. If $\Gamma$ is the subgroup of $G$ defined by this condition, then $\Delta \lhd \Gamma$ and $\mathrm{Aut} C = \Gamma/\Delta$. Now we can see in a very explicit way the short exact sequence above: the inclusion $\mu_2 \to \overline{\mathbb F}_5^\ast$ gives the first map in the short exact sequence, and the projection to the first factor gives the second map.

Moreover, it is now clear that exactly half of the automorphisms will be defined over $\mathbb F_5$, namely those for which $\det \gamma$ is a square in $\mathbb F_5$ (and this depends only on the class of $\gamma$ in $PGL(2,\mathbb F_5)$. This recovers Tim's statement that the automorphism group is an extension of $C_2$ by $A_5 \cong PSL(2,\mathbb F_5)$.

-
 And the involution will induce the -1 on the jacobian? – shenghao Nov 25 2010 at 18:21 Yes, the hyperelliptic involution induces $-1$ on the Jacobian. – Dan Petersen Nov 25 2010 at 18:22 But I'm very confused by why I don't get the same answer as Tim. Certainly the hyperelliptic involution should be defined over $\mathbb F_5$ since it is just given by $y \mapsto -y$. – Dan Petersen Nov 25 2010 at 18:23 Yes, absolutely, it is defined over $F_5$. Can I do a sanity check? Would your argument give that the reduced automorphism group of $y^2=x^3-x$ over $F_3$ is $PGL(2,F_3)$? Or does this work differently for elliptic curves? – Tim Dokchitser Nov 25 2010 at 18:28 Sure, as long as you don't demand that your automorphism fix the origin of your elliptic curve. – Dan Petersen Nov 25 2010 at 18:29

There are quite a few papers counting "isomorphism classes" of hyperelliptic curves following on from a paper of Lockhart. They all restrict to isomorphisms which fix a point at infinity.

If I recall correctly, the paper by L. Hernández Encinas and J. Muñoz Masqué is in that line of work, which is why their result on Aut( C ) will be different.

-

This paper should provide the answer for all automorphism groups of genus 2 curves defined over an algebraically closed field of characteristic not equal to 2.

Shaska, T.; Voelklein, H, Elliptic subfields and automorphisms of genus 2 function fields. Algebra, arithmetic and geometry with applications (West Lafayette, IN, 2000),703--723, Springer, Berlin, 2004.

-