Let $V_\mathbb{R}$ denote the $\mathbb{R}$-vector space of binary quartic forms. The group $\operatorname{GL}_2(\mathbb{R})$ acts on $V_\mathbb{R}$ via the standard substitution action. That is, if $F(x,y) \in V_\mathbb{R}$ and $T = \begin{pmatrix} t_1 & t_2 \\ t_3 & t_4 \end{pmatrix} \in \operatorname{GL}_2(\mathbb{R})$, then $T$ sends $F$ to $F_T(x,y) = F(t_1 x + t_2 y, t_3 x + t_4 y)$. The action induced by the subgroup $\operatorname{GL}_2(\mathbb{Z})$ of $\operatorname{GL}_2(\mathbb{R})$ has two invariants, $I(F)$ and $J(F)$, which are algebraically independent and generate the ring of invariants under the action of $\operatorname{GL}_2(\mathbb{Z})$.

We will denote by the height of $F$, as in Bhargava and Shankar's paper (see references below), as $H(F) = \max\{|I(F)|^3, J(F)^2/4\}$.

The action of $\operatorname{GL}_2(\mathbb{R})$ on $F \in V_\mathbb{R}$ has a stabilizer, which we will denote by $\operatorname{Aut}_\mathbb{R} (F)$.

We denote by $V_\mathbb{Z}$ the subring of $V_\mathbb{R}$ consisting of binary quartic forms with integer coefficients. We will say an element $U \in \operatorname{Aut}_\mathbb{R}(F)$ *almost rational* if it is of the form

$$\displaystyle U = U(\alpha, \beta, \gamma) = \frac{1}{\sqrt{D}} \begin{pmatrix} \beta & 2 \gamma \\ -2 \alpha & -\beta \end{pmatrix},$$

where $\alpha, \beta, \gamma$ are co-prime integers and $D = |\beta^2 - 4 \alpha \gamma|$. We will say that $U$ is *reduced* if $f(x,y) = \alpha x^2 + \beta xy + \gamma y^2$ is a reduced binary quadratic form (in the sense of Gauss).

We will say that a binary quartic form $F$ of height less than $Z$ has a "large" stabilizer if $\operatorname{Aut}_\mathbb{R} (F)$ contains a reduced almost rational element $U(\alpha, \beta, \gamma)$ such that $D \gg Z^{1/6}$. How does one count the number of irreducible forms with "large" stabilizer of height up to $Z$?

There is a reason for the exponent of $1/6$. Consider the following family of forms fixed by the matrix

$$\displaystyle U(1, 0, D) = \frac{1}{\sqrt{D}} \begin{pmatrix} 0 & D \\ -1 & 0 \end{pmatrix}.$$

The family of forms $F$ fixed by $U(1, 0, D)$ are the forms of the shape

$$\displaystyle F(x,y) = a_4 x^4 + a_3 x^3 y + a_2 x^2 y^2 - a_3 Dxy^3 + a_4 D^2 y^4,$$

and the $I$-invariant of the generic form in this family is given by

$$\displaystyle I(F) = 12 a_4^2 D^2 + 3 a_3^2 D + a_2^2.$$

Since for every form $F$ in the family the height $H(F)$ is given by $H(F) = I(F)^3$ (one checks that every form in this family has non-negative discriminant), it follows that the height condition is equivalent to

$$\displaystyle 12a_4^2 D^2 + 3a_3^2 D + a_2^2 \leq Z^{1/3}.$$

Thus, if $D \gg Z^{1/6}$, then $a_4 = 0$, which means that all such forms are reducible, which we do not want to count. However, this argument does not seem to work for an arbitrary family, so I am wondering if there is a more subtle principle at work.

Reference:

M. Bhargava, A. Shankar, *Binary quartic forms having bounded invariants, and the boundedness of the average rank of elliptic curves*, Annals of Mathematics **181** (2015), 191-242.