Return to Answer

Here is a necessary and sufficient condition for $G^{-}(f)$ to be non-empty, taken from [1, Exercise 6.21]:

Let $S = \begin{pmatrix} a & b/2 \\ b/2 & c \end{pmatrix}$ with $a,b$ and $c \in \mathbb{Z}$. Then the following are equivalent:

 

$(1)$ There exists $A \in \text{GL}_2(\mathbb{Z})$ with $\det(A)= -1$ such that $^tASA = S$.

 

$(2)$ The matrix $S$ is $\text{SL}_2(\mathbb{Z})$-equivalent to $\begin{pmatrix} a & -b/2 \\ -b/2 & c \end{pmatrix}$.

 

$(3)$ There exists $S' = \begin{pmatrix} a' & b'/2 \\ b'/2 & c' \end{pmatrix}$ with $a', b'$ and $c' \in \mathbb{Z}$ which is $\text{SL}_2(\mathbb{Z})$-equivalent to $S$ such that $b'$ is divisible by $a'$. (Here we may take $b' = a'$ or $b' = 0$.)

Hint: If $\,^tASA = S$ with $\det(A)= -1$, then show that $\text{Tr}(A) = 0$ and $A^2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$. Then reduce the case when $A$ is upper triangular.

If the above conditions hold, the $\text{SL}_2(\mathbb{Z})$-class to which such an $S$ belongs is called an ambig class (sic). This is also what John Robertson calls an ambiguous class in [2]. From this source, we get

Let $D \in \mathbb{Z}$ which is not a square and such that $D \equiv 0$ or $D \equiv 1 \,(\text{mod } 4)$. Then the number of ambiguous classes is the order of the genus group of $D$, that is, the order of $H^{+}(D) \otimes \mathbb{Z}/2\mathbb{Z}$ where $H^{+}(D)$ is the strict class group of $D$.

[1] T. Ibukiyama, M. Kaneko, "Quadratic Forms and Ideal Theory of Quadratic Fields" in Bernoulli Numbers and Zeta Functions, pp 75-93, 2014.
[2] J. Robertson, "Computing in Quadratic Orders", 2009.

Here is a necessary and sufficient condition for $G^{-}(f)$ to be non-empty, taken from [1, Exercise 6.21]:

Let $S = \begin{pmatrix} a & b/2 \\ b/2 & c \end{pmatrix}$ with $a,b$ and $c \in \mathbb{Z}$. Then the following are equivalent:

$(1)$ There exists $A \in \text{GL}_2(\mathbb{Z})$ with $\det(A)= -1$ such that $^tASA = S$.

$(2)$ The matrix $S$ is $\text{SL}_2(\mathbb{Z})$-equivalent to $\begin{pmatrix} a & -b/2 \\ -b/2 & c \end{pmatrix}$.

$(3)$ There exists $S' = \begin{pmatrix} a' & b'/2 \\ b'/2 & c' \end{pmatrix}$ with $a', b'$ and $c' \in \mathbb{Z}$ which is $\text{SL}_2(\mathbb{Z})$-equivalent to $S$ such that $b'$ is divisible by $a'$. (Here we may take $b' = a'$ or $b' = 0$.)

Hint: If $\,^tASA = S$ with $\det(A)= -1$, then show that $\text{Tr}(A) = 0$ and $A^2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$. Then reduce the case when $A$ is upper triangular.

If the above conditions hold, the $\text{SL}_2(\mathbb{Z})$-class to which such an $S$ belongs is called an ambig class (sic). This is also what John Robertson calls an ambiguous class in [2]. From this source, we get

Let $D \in \mathbb{Z}$ which is not a square and such that $D \equiv 0$ or $D \equiv 1 \,(\text{mod } 4)$. Then the number of ambiguous classes is the order of the genus group of $D$, that is, the order of $H^{+}(D) \otimes \mathbb{Z}/2\mathbb{Z}$ where $H^{+}(D)$ is the strict class group of $D$.

[1] T. Ibukiyama, M. Kaneko, "Quadratic Forms and Ideal Theory of Quadratic Fields" in Bernoulli Numbers and Zeta Functions, pp 75-93, 2014.
[2] J. Robertson, "Computing in Quadratic Orders", 2009.

Here is a necessary and sufficient condition for $G^{-}(f)$ to be non-empty, taken from [1, Exercise 6.21]:

Let $S = \begin{pmatrix} a & b/2 \\ b/2 & c \end{pmatrix}$ with $a,b$ and $c \in \mathbb{Z}$. Then the following are equivalent:

$(1)$ There exists $A \in \text{GL}_2(\mathbb{Z})$ with $\det(A)= -1$ such that $^tASA = S$.

$(2)$ The matrix $S$ is $\text{SL}_2(\mathbb{Z})$-equivalent to $\begin{pmatrix} a & -b/2 \\ -b/2 & c \end{pmatrix}$.

$(3)$ There exists $S' = \begin{pmatrix} a' & b'/2 \\ b'/2 & c' \end{pmatrix}$ with $a', b'$ and $c' \in \mathbb{Z}$ which is $\text{SL}_2(\mathbb{Z})$-equivalent to $S$ such that $b'$ is divisible by $a'$. (Here we may take $b' = a'$ or $b' = 0$.)

Hint: If $\,^tASA = S$ with $\det(A)= -1$, then show that $\text{Tr}(A) = 0$ and $A^2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$. Then reduce the case when $A$ is upper triangular.

If the above conditions hold, the $\text{SL}_2(\mathbb{Z})$-class to which such an $S$ belongs is called an ambig class (sic). This is also what John Robertson calls an ambiguous class in [2]. From this source, we get

Let $D \in \mathbb{Z}$ which is not a square and such that $D \equiv 0$ or $D \equiv 1 \,(\text{mod } 4)$. Then the number of ambiguous classes is the order of the genus group of $D$, that is, the order of $H^{+} \otimes \mathbb{Z}/2\mathbb{Z}$ where $H^{+}(D)$ is the strict class group of $D$.

[1] T. Ibukiyama, M. Kaneko, "Quadratic Forms and Ideal Theory of Quadratic Fields" in Bernoulli Numbers and Zeta Functions, pp 75-93, 2014.
[2] J. Robertson, "Computing in Quadratic Orders", 2009.

Here is a necessary and sufficient condition for $G^{-}(f)$ to be non-empty, taken from [1, Exercise 6.21]:

Let $S = \begin{pmatrix} a & b/2 \\ b/2 & c \end{pmatrix}$ with $a,b$ and $c \in \mathbb{Z}$. Then the following are equivalent:

$(1)$ There exists $A \in \text{GL}_2(\mathbb{Z})$ with $\det(A)= -1$ such that $^tASA = S$.

$(2)$ The matrix $S$ is $\text{SL}_2(\mathbb{Z})$-equivalent to $\begin{pmatrix} a & -b/2 \\ -b/2 & c \end{pmatrix}$.

$(3)$ There exists $S' = \begin{pmatrix} a' & b'/2 \\ b'/2 & c' \end{pmatrix}$ with $a', b'$ and $c' \in \mathbb{Z}$ which is $\text{SL}_2(\mathbb{Z})$-equivalent to $S$ such that $b'$ is divisible by $a'$. (Here we may take $b' = a'$ or $b' = 0$.)

Hint: If $\,^tASA = S$ with $\det(A)= -1$, then show that $\text{Tr}(A) = 0$ and $A^2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$. Then reduce the case when $A$ is upper triangular.

If the above conditions hold, the $\text{SL}_2(\mathbb{Z})$-class to which such an $S$ belongs is called an ambig class (sic). This is also what John Robertson calls an ambiguous class in [2]. From this source, we get

Let $D \in \mathbb{Z}$ which is not a square and such that $D \equiv 0$ or $D \equiv 1 \,(\text{mod } 4)$. Then the number of ambiguous classes is the order of the genus group of $D$, that is, the order of $H^{+}(D) \otimes \mathbb{Z}/2\mathbb{Z}$ where $H^{+}(D)$ is the strict class group of $D$.

[1] T. Ibukiyama, M. Kaneko, "Quadratic Forms and Ideal Theory of Quadratic Fields" in Bernoulli Numbers and Zeta Functions, pp 75-93, 2014.
[2] J. Robertson, "Computing in Quadratic Orders", 2009.

Here is a necessary and sufficient condition for $G_{-}(f)$ to be non-empty, taken from [1, Exercise 6.21]:

Let $S = \begin{pmatrix} a & b/2 \\ b/2 & c \end{pmatrix}$ with $a,b$ and $c \in \mathbb{Z}$. Then the following are equivalent:

$(1)$ There exists $A \in \text{GL}_2(\mathbb{Z})$ with $\det(A)= -1$ such that $^tASA = S$.

$(2)$ The matrix $S$ is $\text{SL}_2(\mathbb{Z})$-equivalent to $\begin{pmatrix} a & -b/2 \\ -b/2 & c \end{pmatrix}$.

$(3)$ There exists $S' = \begin{pmatrix} a' & b'/2 \\ b'/2 & c' \end{pmatrix}$ with $a', b'$ and $c' \in \mathbb{Z}$ which is $\text{SL}_2(\mathbb{Z})$-equivalent to $S$ such that $b'$ is divisible by $a'$. (Here we may take $b' = a'$ or $b' = 0$.)

Hint: If $\,^tASA = S$ with $\det(A)= -1$, then show that $\text{Tr}(A) = 0$ and $A^2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$. Then reduce the case when $A$ is upper triangular.

If the above conditions hold, the class to which such an $S$ belongs is called an ambig class (sic).

Disclaimer. I am aware this is likely not to be the algorithmic description the OP is looking for, and it is essentially a rewording of Will Jagy's answer. Still, I think this helps seize the problem.

[1] T. Ibukiyama, M. Kaneko, "Quadratic Forms and Ideal Theory of Quadratic Fields" in Bernoulli Numbers and Zeta Functions, pp 75-93, 2014.

Here is a necessary and sufficient condition for $G^{-}(f)$ to be non-empty, taken from [1, Exercise 6.21]:

Let $S = \begin{pmatrix} a & b/2 \\ b/2 & c \end{pmatrix}$ with $a,b$ and $c \in \mathbb{Z}$. Then the following are equivalent:

$(1)$ There exists $A \in \text{GL}_2(\mathbb{Z})$ with $\det(A)= -1$ such that $^tASA = S$.

$(2)$ The matrix $S$ is $\text{SL}_2(\mathbb{Z})$-equivalent to $\begin{pmatrix} a & -b/2 \\ -b/2 & c \end{pmatrix}$.

$(3)$ There exists $S' = \begin{pmatrix} a' & b'/2 \\ b'/2 & c' \end{pmatrix}$ with $a', b'$ and $c' \in \mathbb{Z}$ which is $\text{SL}_2(\mathbb{Z})$-equivalent to $S$ such that $b'$ is divisible by $a'$. (Here we may take $b' = a'$ or $b' = 0$.)

Hint: If $\,^tASA = S$ with $\det(A)= -1$, then show that $\text{Tr}(A) = 0$ and $A^2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$. Then reduce the case when $A$ is upper triangular.

If the above conditions hold, the $\text{SL}_2(\mathbb{Z})$-class to which such an $S$ belongs is called an ambig class (sic). This is also what John Robertson calls an ambiguous class in [2]. From this source, we get

Let $D \in \mathbb{Z}$ which is not a square and such that $D \equiv 0$ or $D \equiv 1 \,(\text{mod } 4)$. Then the number of ambiguous classes is the order of the genus group of $D$, that is, the order of $H^{+} \otimes \mathbb{Z}/2\mathbb{Z}$ where $H^{+}(D)$ is the strict class group of $D$.

[1] T. Ibukiyama, M. Kaneko, "Quadratic Forms and Ideal Theory of Quadratic Fields" in Bernoulli Numbers and Zeta Functions, pp 75-93, 2014.
[2] J. Robertson, "Computing in Quadratic Orders", 2009.