On the automorphism group of binary quadratic forms

Question

This question is a continuation of the following two questions:

Discriminants of indefinite integral binary quadratic forms admitting 3 or 6 torsion.

On certain solutions of a quadratic form equation

Let $f$ be a binary quadratic form with integer coefficients and non-zero discriminant. For $T = \begin{pmatrix}t_1 & t_2 \\ t_3 & t_4 \end{pmatrix} \in \operatorname{GL}_2(\mathbb{R})$, define $f_T(x,y) = f(t_1 x + t_2 y, t_3 x + t_4 y)$. Put

$$\displaystyle \operatorname{Aut}(f) = \{T \in \operatorname{GL}_2(\mathbb{Z}) : f_T = f\}.$$

When $f$ is positive definite, then $\# \operatorname{Aut}(f)$ is easy to determine. In particular, if $f(x,y)$ is reduced, so that it is written as

$$\displaystyle f(x,y) = ax^2 + bxy + cy^2$$

with $|b| \leq a \leq c$ with $ac > 0$, then $\# \operatorname{Aut}(f) = 12$ when $a = b = c = 1$, $\# \operatorname{Aut}(f) = 8$ when $a = c = 1, b = 0$, $\# \operatorname{Aut}(f) = 4$ when $a = \pm b$, and $\#\operatorname{Aut}(f) = 2$ otherwise. When $f$ is reducible, the situation is also easy to account for. The most difficult case is when $f$ is indefinite and irreducible.

One part of the problem is easy. If we consider only positive determinant elements, then they are given by

$$\displaystyle \begin{pmatrix} \dfrac{s+bt}{2} & & -ct \\ \\ at & & \dfrac{s - bt}{2} \end{pmatrix},$$

where $(s,t)$ is a solution to the Pell equation $x^2 - (b^2 - 4ac)y^2 = 4$ (note that this equation is always soluble since $(2,0)$ is a solution).

The problem is the elements of negative determinant. I can show (see the second question linked above) that such elements take the form

$$\displaystyle \begin{pmatrix} m & (bm + cn)/a \\ n & - m\end{pmatrix},$$

where $(m,n)$ is a solution to $am^2 + bmn + cn^2 = a$, which is again always soluble. But there is no guarantee that $(bm+cn)/a$ is an integer. This is guaranteed if $n$ is co-prime to $a$, since then looking at the equation above we see that $a | n(bm+cn)$, whence $a | bm + cn$. There are however examples where this doesn't happen; I believe $5x^2 + 3xy - 11y^2$ is such an example. All of the solutions to $5x^2 + 3xy - 11y^2 = 5$ have $5 | y$, and since $5 \nmid 3$ and $\gcd(m,n) = 1$ by necessity, it is impossible for $3m-11n$ to be divisible by 5 and thus no way for the above matrix to be integral.

However, such examples seem extremely rare. I don't have a lot of computational evidence except to say that there is no such example with $D = b^2 - 4ac < 100$.

The question is, when is the set

$$\displaystyle G^-(f) = \{T \in \operatorname{Aut}(f) : \det T = -1\}$$

non-empty? There are several sufficient conditions for this to be non-empty: if $f$ is equivalent to a diagonal form, if $f$ is equivalent to a form such that $a | b$, , if $f$ is equivalent to a symmetric form, or $f$ is equivalent to a form such that there exists a solution $(m,n)$ to $am^2 + bmn + cn^2 = a$ with $\gcd(n,a) = 1$. Moreover, if the class number is one, then all forms with discriminant $d$ are equivalent and therefore must be equivalent to either $x^2 - (d/4)y^2$ if $d$ is even or $x^2 + xy - (d-1)y^2/4$ if $d$ is odd. Both cases have non-empty $G^-$.

A related question is, suppose that $f_1(x,y) = a_1 x^2 - b_1 y^2$ and $f_2(x,y) = a_2 x^2 - b_2 y^2$ are two non-proportional binary quadratic forms such that $a_1 b_1 = a_2 b_2 > 0$. Then when can $f_1, f_2$ be $\operatorname{GL}_2(\mathbb{Z})$-equivalent?

Answer 1

Here is a necessary and sufficient condition for $G^{-}(f)$ to be non-empty, taken from [1, Exercise 6.21]:

Let $S = \begin{pmatrix} a & b/2 \\ b/2 & c \end{pmatrix}$ with $a,b$ and $c \in \mathbb{Z}$. Then the following are equivalent:

$(1)$ There exists $A \in \text{GL}_2(\mathbb{Z})$ with $\det(A)= -1$ such that $^tASA = S$.

$(2)$ The matrix $S$ is $\text{SL}_2(\mathbb{Z})$-equivalent to $\begin{pmatrix} a & -b/2 \\ -b/2 & c \end{pmatrix}$.

$(3)$ There exists $S' = \begin{pmatrix} a' & b'/2 \\ b'/2 & c' \end{pmatrix}$ with $a', b'$ and $c' \in \mathbb{Z}$ which is $\text{SL}_2(\mathbb{Z})$-equivalent to $S$ such that $b'$ is divisible by $a'$. (Here we may take $b' = a'$ or $b' = 0$.)

Hint: If $\,^tASA = S$ with $\det(A)= -1$, then show that $\text{Tr}(A) = 0$ and $A^2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$. Then reduce the case when $A$ is upper triangular.

If the above conditions hold, the $\text{SL}_2(\mathbb{Z})$-class to which such an $S$ belongs is called an ambig class (sic). This is also what John Robertson calls an ambiguous class in [2]. From this source, we get

Let $D \in \mathbb{Z}$ which is not a square and such that $D \equiv 0$ or $D \equiv 1 \,(\text{mod } 4)$. Then the number of ambiguous classes is the order of the genus group of $D$, that is, the order of $H^{+}(D) \otimes \mathbb{Z}/2\mathbb{Z}$ where $H^{+}(D)$ is the strict class group of $D$.

---

[1] T. Ibukiyama, M. Kaneko, "Quadratic Forms and Ideal Theory of Quadratic Fields" in Bernoulli Numbers and Zeta Functions, pp 75-93, 2014.
[2] J. Robertson, "Computing in Quadratic Orders", 2009.

Answer 2

This may or may not be what you want... my sense is that the conditions you describe as sufficient are also necessary, in this case the bit about equivalent to a form $\langle a,b,c \rangle$ such that $a | b.$ This is from Dickson's Introduction to the Theory of Numbers, 1929. Well, you can judge for yourself.

Answer 3

Just illustrating Dickson's point, that ambiguous forms occur in the cycle of reduced forms of a given form

jagy@phobeusjunior:~/old drive/home/jagy/Cplusplus$ ./indefCycle 5 0 -17

  0  form              5           0         -17  delta      0
  1  form            -17           0           5  delta      1
  2  form              5          10         -12


          -1          -1
           0          -1

To Return  
          -1           1
           0          -1

0  form   5 10 -12   delta  -1     ambiguous  
1  form   -12 14 3   delta  5
2  form   3 16 -7   delta  -2
3  form   -7 12 7   delta  2
4  form   7 16 -3   delta  -5
5  form   -3 14 12   delta  1
6  form   12 10 -5   delta  -2
7  form   -5 10 12   delta  1     ambiguous            -1 composed with form zero  
8  form   12 14 -3   delta  -5
9  form   -3 16 7   delta  2
10  form   7 12 -7   delta  -2
11  form   -7 16 3   delta  5
12  form   3 14 -12   delta  -1
13  form   -12 10 5   delta  2
14  form   5 10 -12


  form   5 x^2  + 10 x y  -12 y^2 

minimum was   3rep   x = -1   y = -1 disc 340 dSqrt 18  M_Ratio  12.96
Automorph, written on right of Gram matrix:  
-130789  -371952
-154980  -440749
=========================================
jagy@phobeusjunior:~/old drive/home/jagy/Cplusplus$

For your example, it is not ambiguous

jagy@phobeusjunior:~/old drive/home/jagy/Cplusplus$ ./indefCycle 5 3 -11

  0  form              5           3         -11  delta      0
  1  form            -11          -3           5  delta      1
  2  form              5          13          -3


          -1          -1
           0          -1

To Return  
          -1           1
           0          -1

0  form   5 13 -3   delta  -4
1  form   -3 11 9   delta  1
2  form   9 7 -5   delta  -2
3  form   -5 13 3   delta  4          -1 composed with form zero  
4  form   3 11 -9   delta  -1
5  form   -9 7 5   delta  2
6  form   5 13 -3


  form   5 x^2  + 13 x y  -3 y^2 

minimum was   3rep   x = 0   y = 1 disc 229 dSqrt 15  M_Ratio  9
Automorph, written on right of Gram matrix:  
-16  -45
-75  -211
=========================================
jagy@phobeusjunior:~/old drive/home/jagy/Cplusplus$