A heuristic for the density of solutions to Diophantine equations

Question

Let $f\in\mathbb{Z}[X_1,\ldots,X_n]$ be a Diophantine equation which, for the purposes of this question, I will assume is homogeneous and nonsingular on $\mathbb{R}^n\setminus\{0\}$ (so that $\nabla f\not=0$). Supposing that it has infinitely many primitive integer zeros, we can posit that they are smoothly distributed in an asymptotic sense. Writing $V(R)\subseteq R^n$ for the set of primitive solutions to $f(x)=0$ in a ring $R$, the integer solutions $V(\mathbb{Z})$ clearly lie on the manifold $V(\mathbb{R})$. So, I am looking for a density $\rho\colon V(\mathbb{R})\to\mathbb{R}$ with $$ \vert V(\mathbb{Z})\cap U\vert\sim\int_{V(\mathbb{R})\cap U}\rho(x)\,d\sigma(x),\qquad\qquad{\rm(1)} $$ for subsets $U\subseteq\mathbb{R}^n$, where $d\sigma$ is the standard surface integral on $V(\mathbb{R})$. This should hold asymptotically as $U$ is scaled up, and for reasonably regular regions $U$.

My question is regarding a simple (but incorrect -- see below) heuristic argument for calculating $\rho$. Choosing positive integer $N$ and real $a\gg N$ then, for large regions $U$, the set of $x\in U$ with $\vert f(x)\vert < 2a$ has volume about $2a\int_{V(\mathbb{R})\cap U}\Vert\nabla f\Vert^{-1}\,d\sigma$, so should contain about that number of integer points. The probability of a random $x\in\mathbb{Z}^n$ being relatively prime to $N$ and satisfying $f(x)=0$ (mod $N$) is $N^{-n}\vert V(\mathbb{Z}/N\mathbb{Z})\vert$. Conditional on $\vert f(x)\vert < 2a$ and $f(x)=0$ (mod $N$), it seems reasonable to suppose that $f(x)=0$ with probability $N/(2a)$. Multiplying these terms together and taking the limit as $N$ increases to include all prime-powers as factors, we get the following expression for $\rho$. $$ \begin{align} &\rho(x)=\Vert\nabla f(x)\Vert^{-1}\prod_p c_p,\qquad\qquad{\rm(2)}\\ &c_p=\lim_{r\to\infty}p^{-r(n-1)}\left\lvert V(\mathbb{Z}/p^r\mathbb{Z})\right\rvert. \end{align} $$ The product is taken over all primes $p$. This seems like a very neat expression, and can be seen that it gives the correct result for linear equations. However, it is not correct in general. Just looking at quadratic forms for $f$, the expression given by (2) is wrong. I do not have any good feeling as to where exactly this heuristic goes astray, and if it is possible to fix it. Maybe this approach and the reason that it does not quite work is well known. This is not an area in which I am any kind of expert, so maybe others on MathOverflow would be able to help?

For example, consider $f=x^2+y^2-z^2$, so that we are looking for primitive Pythagorean triples. Euclid's parameterization $(x,y,z)=(a^2-b^2,2ab,a^2+b^2)$ can be used to show that $\rho=\sqrt{2}\pi^{-2}\vert z\vert^{-1}$. However, on $V(\mathbb{R})$ we have $\Vert\nabla f\Vert = 2\sqrt{2}\vert z\vert$ and you can calculate $c_2=1$ and $c_p=1-p^{-2}$ for odd prime $p$. Using (2) would lead to $\rho=2\sqrt{2}\pi^{-2}\vert z\vert^{-1}$, which is out by exactly a factor of 2. If we look at Pythagorean quadruples $f=w^2+x^2+y^2-z^2$ instead, then we can calculate $c_p=(1-p^{-1})(1+2p^{-1}1_{\{p\equiv1{\rm\ mod\ }4\}}+p^{-2})$ for odd primes $p$, so the product in (2) is not unconditionally convergent.

Is there a known or, even, just conjectural expression for the asymptotic density $\rho$? And, is it possible to explain precisely how the heuristic used to derive (2) fails?

It would be great if my expression (2) above could be fixed. Heuristics like the one used here are often very useful to understand what the integer solutions to Diophantine equations look like, and it is a bit worrying that it gives the wrong answer in this case. It is also consistent with the idea that find rational solutions to an equation, you should first check for solutions in the completions of $\mathbb{Q}$, according to the Hasse principle. Also, it so nearly works (only being a factor of 2 out for Pythagorean triples) and gives perfectly sensible looking results in many cases, that I am loath to give up and just accept that it doesn't work without a good reason as to why. For example, it does seem perfectly consistent with Falting's theorem (as given in my answer to a previous MO question) and with the Birch and Swinnerton-Dyer conjecture. In the case where $f$ is a a cubic describing an elliptic curve, then $c_p=(1-p^{-1})N_p/p$ for all but finitely many primes $p$, where $N_p$ is the number of $\mathbb{F}_p$-points on the elliptic curve reduced modulo $p$. Then, up to finitely many terms, $\prod_pc_p$ coincides with the Euler product at $s=1$ of $(L(s)\zeta(s))^{-1}$, where $L$ is the L-function of the curve. According to the Birch and Swinnerton-Dyer conjecture, I would expect this to be zero, finite, or infinite when the curve has rank $r=0$, $r=1$ and $r>1$ respectively. Putting this back into (2) is consistent with $\vert V(\mathbb{Z})\cap B_R\vert$ growing at rate $(\log R)^r$, which you would expect for an elliptic curve of rank $r$.

Answer 1

You are on the way to redeveloping the singular series, which does indeed give the correct asymptotic for integral solutions to many flavors of Diophantine equation -- they key words here are "Hardy-Littlewood method" or "circle method," which you can read about in any text on analytic number theory, such as the book of Iwaniec and Kowalski.

Loosely speaking -- when the number of variables is very large relative to the degree of the equation, the singular series is known to give you the right asymptotic. When the number of variables is somewhat large relative to the degree of the equation, it is expected to give the right asymptotic but there are no proofs outside very special cases.

Answer 2

In 1989, Manin and his collaborators formed a series of conjectures on the asymptotic behaviour of the number of solutions to diophantine equations. Let $X\subset \mathbb{P}^n$ be a fano variety (that is $-K_X$ is ample) under its anticanonical embedding, and let $H$ be the associated height function. Then it is expected that there exists a zariski open subset $U \subset X$ such that the number of rational points of height less than $B$ (e.g. the number of solutions in an expanding ball or box) is asymptotic to $c_X B(\log B)^{r_X},$ as $B \to \infty$ for some constants $c_X$ and $r_X$.

This result is true in some cases, for example complete intersections with many variables and small degree via the circle method, quadratic forms, toric varieties, flag varieties and also for some del Pezzo surfaces. But there are counter-examples showing that it is not true for all fano varieties (namely one expects $r_X=\textrm{rank } \textrm{Pic}(X)-1$, but this is not true in general).

At any rate, Peyre formed a conjecture on the leading constant $c_X$ which occurs in the asymptotic formula. It is very close to what you describe. One defines a measure on the set of adelic points on $X$, and then the leading constant is essentially the volume of the closure of $X(\mathbb{Q})$ inside the adeles. This is really an adelic integral and not a real integral, but for suitable varieties (namely those which satisfy weak approximation), the local factors at the primes come out as the $c_p$ in the way you describe. In general though one needs to introduce convergence factors to insure that the product over the $c_p$ converges. These come from an Artin L-function associated to the Picard group.

There are however some extra factors $\alpha$ and $\beta$ present in the constant, related to the position of the anticanonical divisor in the effective cone and the Brauer group of $X$. For conics with a rational point, we have $\alpha=1/2$ and $\beta =1$. This might explain your missing factor of two.

Papers:

J. Franke, Y. I. Manin and Y. Tschinkel, Rational Points of Bounded Height on Fano Varieties. Invent. Math. 95, 421--435 (1989).

E. Peyre, Hauteurs et measures de Tamagawa sur les variétiés de Fano. Duke Math. J., 79(1), 101--218 (1995).

Answer 3

Searching including the key phrases "Hardy-Littlewood circle method" and "singular series", as suggested by the other answers, turned up some interesting references which shed light on the question and why I obtained the results I did for the cases mentioned. As the question is already quite long and would become rather unmanageable to add this as a large update, I'm adding it as an answer here.

The product $\prod_pc_p$ is indeed called the singular series and $\int\Vert\nabla f\Vert^{-1}\,d\sigma$ is the singular integral. The product does asymptotically give the number of solutions subject to hypotheses and/or conditions on $f$, generally seeming to work better when the number of variables is large relative to the degree. For quadratic forms, the paper A new form of the circle method, and its application to quadratic forms by D.R. Heath-Brown (Journal für die reine und angewandte Mathematik, 1996. Preprint available here) gives expressions for the asymptotic density which show why I obtained the results I did for Pythagorean triples and quadruples mentioned in the question. For a quadratic form $f$ in $n$ variables, they show the following.

• For $n\ge5$, expression (2) for the asymptotic density given in the question is correct, so the heuristic works! (Theorem 5 of the D.R. Heath-Brown paper).
• For $n=4$ and the determinant of $f$ not a perfect square, you should multiply the terms $c_p$ by $1-\chi(p)p^{-1}$ and the overall expression by $L(1,\chi)$ (Theorem 6 of the paper). Here, the character $\chi$ is the Jacobi symbol $\left({\rm det}(f)\over\ast\right)$. Leaving out these terms will give a product which is not unconditionally convergent, as happened for Pythagorean quadruples.
• For $n=4$ and the determinant of $f$ a perfect square, the product $\prod_pc_p$ will diverge to infinity (assuming that there are solutions in every $p$-adic field, so $c_p\not=0$). Instead, $c_p$ should be multiplied by $1-p^{-1}$ and the overall density by $\log\Vert x\Vert$ (Theorem 7 of the paper).
• For $n=3$ then expression (2) given in the question works after multiplying by a factor of $\frac12$ (Theorem 8 and Corollary 2 of the paper)! This is why I was out by a factor of 2 for Pythagorean triples. The D.R. Heath-Brown paper has the following to say on this.

...It therefore remains to understand the appearance of the factor $\frac12$ in the case $n=3$, which can be thought of as corresponding to a Tamagawa number of 2. In the proof of Theorem 8 this factor arises from the residue at s = 0 of $$\zeta(2s+1)\frac{P^s}{s}.$$

I'm not very familiar with Tamagawa numbers and am not yet sure whether this is the same as the factor $\alpha=\frac12$ mentioned in Daniel's answer or how it comes into the heuristic derivation.