Let $p,q$ be a pair of coprime positive integers. Let $S(N,p,q)$ be the number of integer solutions $(x,y)$ of $N=p x^2+q y^2$ such that $x$ and $y$ are coprime. If $(p,q)=(1,1)$, it follows from Fermat's two square theorem that $S(N,1,1)$ has no upper bound as $N\to\infty$. What will happens if $(p,q)\neq(1,1)$, i.e. is the sequence $\{S(N,p,q)\}_{N\geq1}$ still unbounded? Does there exist $(p,q)$ such that $\{S(N,p,q)\}_{N\geq1}$ is a bounded sequence?
Let $F$ be a positive definite binary quadratic form. For a positive number $Z$ put $N_F(Z)$ for the number of pairs of integers $(x,y)$ such that $F(x,y) \leq Z$, and let $R_F(Z)$ denote the number of integers $n \leq Z$ such that the equation $F(x,y) = n$ has solutions in integers $x$ and $y$. It is easy to see from a geometry of numbers argument that
$$\displaystyle N_F(Z) \sim A_F Z$$
for a positive number $A_F$, and it is known (first by Landau in the case $F(x,y) = x^2 + y^2$ and in general by the thesis of one of his students) that there exists a positive number $C_F$ such that
$$\displaystyle R_F(Z) \sim C_F Z (\log Z)^{1/2}.$$
From these asymptotic formulas one can conclude, in your notation, that $S(n,p,q)$ behaves like $\sqrt{\log n}$ on average; in particular, it is unbounded.

$\begingroup$ Thanks! Is there any reference for the results of Landau and his student? $\endgroup$ – Huiping Pan Feb 2 '18 at 12:47