Is there anything special about the Riemann surface $y^2 = x(x^{10}+11x^5-1)$?

Question

I stumbled upon the fact that the Bolza surface can be obtained as the locus of the equation,

$$y^2 = \color{blue}{x^5-x}.$$

Its automorphism group has the highest order for genus $2$, namely $48$. I recognized $x^5-x$ as a polynomial invariant of the octahedron. In fact, the Bolza surface is connected to the octahedron.

(Edit, Apr 27, 2023:) Since someone has bumped this to the front page, the connection to the octahedron can be illustrated by the j-function formula,

$$j(\tau)-1728 = \frac{16(x^8 + 14x^4 + 1)^3}{(\color{blue}{x^5 - x})^4}-1728 = \frac{16(x^{12} - 33x^8 - 33x^4 + 1)^2}{(\color{blue}{x^5 - x})^4}$$

where the numerator of the RHS square is an invariant of the octahedral group $H_{48}$. (Dickson, p. 230, Algebraic Theories, 1959.) The factorization $\color{blue}{x^5-x} = x(x^2-1)(x^2+1)$ involving Pythagorean triples is of course familiar, but not so familiar is it can express the other numerator,

$$(x^2-1)^4+(2x)^4+(x^2+1)^4 = 2(x^8+14x^4+1)$$ $$\;(x^2-1)^8+(2x)^8+(x^2+1)^8 = 2(x^8+14x^4+1)^2$$

---

Question:

If we use the analogous polynomial invariant of the icosahedron, then does the genus 5 surface,

$$y^2 = x(x^{10}+11x^5-1)$$

have special properties? How close does the order of its automorphism group get to the bound $84(g-1)$? (For $g = 5$, this would be $336$.)

---

POSTSCRIPT:

My thanks to Noam Elkies for the highly detailed answer below. The background to this question is an identity I found involving $x^{10}+11x^5-1$. Define,

$$a = \frac{r^5(r^{10}+11r^5-1)^5}{(r^{30}+522r^{25}-10005r^{20}-10005r^{10}-522r^5+1)^2}$$

and,

$$w = \frac{r^2(r^{10}+11r^5-1)^2(r^6+2r^5-5r^4-5r^2-2r+1)}{r^{30}+522r^{25}-10005r^{20}-10005r^{10}-522r^5+1}$$

then they can be neatly stitched together as,

$$w^5-10aw^3+45a^2w-a^2 = 0$$

for arbitrary $r$. This in fact is the Brioschi quintic form which the general quintic can be reduced into. Two of the polynomials are easily recognizable as icosahedral invariants, while $r^6+2r^5-5r^4-5r^2-2r+1$ is a factor of $r^{30}+522r^{25}-10005r^{20}-10005r^{10}-522r^5+1$.

So other than in formulas using Ramanujan's continued fractions, I wondered where else those polynomials appear. Since the Bolza surface involved an invariant of the octahedron, it was reasonable to consider if using the corresponding one for the icosahedron would also be special. As Elkies wonderfully showed, it turns out that it is.

Answer 1

Yes, this Riemann surface, call it $C: y^2 = x^{11}-11x^6-x$, is quite special: not only does it have the maximal number of automorphisms for a hyperelliptic surface of genus $5$, but it is a modular curve in at least two ways, both of which exhibit its full automorphism group.

One is a classical (elliptic) modular curve of level $10$, intermediate between $X(5)$ and $X(10)$, with $[C:X(5)] = 2$ (the hyperelliptic map) and $[X(10):C] = 3$ (a cyclic cover); this modular curve parametrizes elliptic curves $E$ with full level-$5$ structure and odd ${\rm Gal}(E[2])$, or equivalently full level-$5$ structure and square $j(E)-12^3$. Explicitly, $E$ has Weierstrass equation $Y^2 = X^3 - A(x)X/48 + B(x)/864$ where $A(x) = x^{20} + 228x^{15} + 494x^{10} - 228x^5 + 1$ and $$ x^{30} - 522x^{25} - 10005x^{20} - 10005x^{10} + 522x^5 + 1 $$ are polynomials with roots at the $20$- and $30$-point orbits of $A_5$. We have $A^3 - B^2 = 12^3 (x^{11}-11x^6-x)^5$, so $j - 12^3 = B^2/(x^{11}-11x^6-x)^5$. The corresponding congruence subgroup $\Gamma$ of ${\rm SL}_2({\bf Z})$ is the index-$2$ subgroup of $\Gamma(5)$ consisting of matrices that reduce mod $2$ to the index-$2$ subgroup of ${\rm SL}_2({\bf Z}/2{\bf Z})$, with $[\Gamma : \Gamma(10)] = 3$. This $\Gamma$ is normal in ${\rm SL}_2({\bf Z})$, and the quotient group is ${\rm Aut}(C)$.

Another modular approach to $C$ is via the $(2,3,10)$ triangle group, call it $G^*$, which appears in class VIII of the nineteen commensurability classes tabulated in

Takeuchi, K.: Commensurability classes of arithmetic triangle groups, J. Fac. Sci. Univ. Tokyo 24 (1977), 201-212.

According to Takeuchi's table, $G^*$ is the normalizer of the unit-norm group $G_1$ of a maximal order in a quaternion algebra over ${\bf Q}(\sqrt 5)$ ramified over one real place and the prime $(\sqrt 5)$.
Moreover $G_1$ is the $(3,3,5)$ triangle group, contained in $G^*$ with index $2$. Let $G_5$ be the normal subgroup of $G_1$ consisting of units congruent to $1 \bmod (\sqrt 5)$. Then $G^*/G_5 \cong \lbrace \pm 1 \rbrace \times A_5$, and the quotient of the upper half plane $\cal H$ by $G_5$ has genus $5$, so must be our $C$. Moreover, ${\cal H} / G_5$ has no elliptic points, so this identifies the image of the fundamental group $\pi_1(C)$ in ${\rm Aut}{\cal H} = {\rm SL}_2({\bf R})$ with an arithmetic congruence group.

P.S. Roy Smith already noted that if we allow also non-hyperelliptic Riemann surfaces then the maximal number of automorphisms for genus $5$ is not $120$ but $192$. An explicit model for a Riemann surface $S$ with $192$ automorphisms is the intersection of three quadrics $$ y^2 = x_0 x_1, \phantom{and} {y'}^2 = x_0^2 - x_1^2, \phantom{and} {y''}^2 = x_0^2 + x_1^2 $$ in ${\bf P}^4$. Then $(x_0:x_1:y:y':y'') \mapsto (x_0:x_1)$ gives a normal cover $S \rightarrow {\bf P}^1$ with Galois group $N = ({\bf Z}/2{\bf Z})^3$ acting by arbitrary sign changes on $y,y',y''$, ramified above the vertices of a regular octahedron, with each of $x_0 x_1, x_0^2 - x_1^2, x_0^2 + x_1^2$ vanishing on an opposite pair of vertices. I claim that there is an exact sequence $1 \rightarrow N \rightarrow {\rm Aut}(S) \rightarrow S_4 \rightarrow 1$, so in particular $\#({\rm Aut}(S)) = 2^3 4! = 192$. Indeed let $G$ be the subgroup of ${\rm Aut}(S)$ that stabilizes the span of $\lbrace x_0, x_1 \rbrace$. Then $G$ contains $N$ as the kernel of a homomorphism $G \rightarrow {\rm Aut}({\bf P}^1)$ given by the action on $(x_0:x_1)$. The image is contained in the group $S_4$ of rotations of the octahedron, and indeed equals $S_4$ because any rotation permutes the three opposite pairs of vertices and thus lifts to ${\rm Aut}(S)$. Therefore ${\rm Aut}(S)$ contains a group $G$ of order $2^3 4! = 192$, and by the Hurwitz bound this must be the full group of automorphisms, QED

Answer 2

Your curve is hyperelliptic.

If $X_g$ is a hyperelliptic curve of genus $g$, then $\textrm{Aut}(X_g)$ is a central extension of degree $2$ of one of the groups $$\mathbb{Z}_n, D_n, A_4, S_4, A_5,$$

see Shaska - Determining the automorphism group of a hyperelliptic curve.

In the case of Bolza curve the polynomial $x^5-x$ is invariant by the automorphism group of the octahedron, which is $S_4$. In fact, the automorphism group of the Bolza curve is a central extension of $S_4$ by the group of order $2$ generated by the hyperelliptic involution, hence it has order $2 \cdot |S_4|=48$.

Regarding your curve, the polynomial at the right hand side is invariant by the automorphism group of the icosahedron, which is $A_5$. Then the automorphism group is a central extension of $A_5$ by the hyperelliptic involution, hence it has order $2 \cdot |A_5|= 120$.