# Upper bounds for lattice points in orbits, and representations of binary quadratic forms

Write $$\mathbb{Z}^{a\times b}$$ for the $$a\times b$$ integer matrices. Let $$n\geq 3$$ and $$Q\in\mathbb{Z}^{n\times n}$$. Let $$G=O(Q)$$ be the orthogonal group of $$Q$$. For $$X_0\in \mathbb{Z}^{2\times n}$$, set $$R'(T,Q,X_0) = \{ X\in X_0G(\mathbb{Z}) : |X_{ij}|\leq T\}.$$

Dubious heuristic: If $$Q$$ is indefinite then $$\#R'(T,Q,X_0) \ll_{Q,\epsilon} T^{\max\{0,2n-6\}+\epsilon} \tag{\star}$$ for all $$\epsilon>0$$, with an implicit constant depending only on $$Q$$ and $$\epsilon$$.

Question: For which $$Q$$ and which $$X_0$$ is this true? How about simple (?) examples like $$Q=\operatorname{diag}(1,1,-1)$$ or $$Q=\operatorname{diag}(1,1,-1,-1)$$?

Update 2018/4/11: I wonder if the work of Gorodnik and Nevo allows one to prove upper and lower bounds of the form $$R'(T,Q,X_0)\asymp_Q T^{\text{something}}$$ for all $$X_0$$ with entries $$\ll T^{\delta}$$, for some small $$\delta$$. In particular this would answer the question for all small $$X_0$$. I will try to look into this further.

Update 2018/4/11: As he explains in his answer below, Emilio Lauret gave in his PhD thesis a general strategy to answer questions of this type. This uses the result of Gorodnik and Nevo mentioend above. There is nontrivial work still to be done to implement this strategy in any particular case, more specifically one must compute the constant $$\delta$$ from page 45, line -5 of his thesis. (See his answer for a link). Unless and until someone comes up with an ever better approach to the problem, I accept his answer!

## Background

Let $$B\in\mathbb{Z}^{2\times 2}$$ be nonsingular and symmetric. I am interested in the number of solutions $$R(T,Q,B) = \{ X\in \mathbb{Z}^{2\times n} : XQX^T =B, \, |X_{ij}|\leq T\}$$ with height up to $$T$$, when $$Q$$ is indefinite. A naive guess based on the circle method would be that $$\#R(T,Q,B) \ll_Q 1+T^{2n-6} \tag{\ast}$$ with an implicit constant depending at most on $$Q$$. The set $$R(T,Q,B)$$ is the union of finitely many $$G(\mathbb{Z})$$-orbits $$R'(T,Q,X_0)$$, so the conjecture above is a weak version of this.

Note that ($$\ast$$) is true for large $$n$$. For instance when $$Q$$ is indefinite and $$n\geq 12$$ this (and much more) follows from the asymptotic result of Brandes. (The condition $$n\geq 12$$ follows from the comment at the end of section 1 in the subsequent paper.) On the other hand ($$\ast$$) is certainly false for small $$n$$, though I don't know of a good example in the indefinite case.

If $$B$$ is positive definite and $$Q$$ has signature $$(p,q)$$ with $$p\geq q$$ then it is a special case of Theorem 1.4 of Eskin, Mozes and Shah that $$\# R(T,Q,B) \sim C T^{\min\{2,q\}(n-1-\min\{2,q\})} \tag{\dagger}$$ for some $$C= C(Q,B)$$, unless a certain volume is infinite. In the latter case, the comments after formula (1.4) of Duke, Rudnick and Sarnak suggest that the right-hand side of ($$\dagger$$) just changes by a factor of $$\log T$$. If this is correct then ($$\star$$) can be false in the case when $$(p,q)=(2,1)$$ and $$B=X_0Q X_0^T$$ is positive definite, since $$\# R(T,Q,B)\gg T$$ for all $$T\gg_{Q,B} 1$$.

Perhaps a proof or disproof of ($$\star$$) in other cases is implicit in the work cited in the previous paragraph, but I have not been able to verify this.

This is not an answer, but a long comment providing related known results.

In the particular case $$Q=I_{n,1}= diag(I_n,-1)=diag(1,\dots,1,-1)$$ (called the Lorentzian case), Ratcliffe and Tschantz gave in 1 an asymptotic formula for the number of $$x=(x_1,\dots,x_{n+1})^t\in \mathbb Z^{n+1}$$ satisfying that $$x^tI_{n,1}x=k$$ ($$k\in\mathbb Z$$) and $$\|x\|^2\leq t$$, as $$t\to\infty$$. When $$k$$ is negative, they applied (in a clever way) the lattice point theorem by Lax and Phillips, obtaining a formula with an error term.

In my Ph.D. thesis (available at my web page, though it is written in Spanish) I extended the above formulas for more general indefinite quadratic or hermitian forms $$Q$$ of signature $$(n,1)$$ (see ). Furthermore, I also worked on counting certain solutions of $$X\in \mathbb Z^{(p+q)\times q}$$ satisfying $$I_{p,q}[X]=X^*I_{p,q}X=-L$$, where $$L$$ is a positive definite integral matrix by using a lattice point theorem by Gorodnik and Nevo  or . Unfortunately, concerning the last case, it is not easy to compute explicitly the constants involved in the asymptotic formulas.

I hope similar technics as above may help to find upper bounds like those requested in the question.

1 Ratcliffe, John G.; Tschantz, Steven T., On the representation of integers by the Lorentzian quadratic form, J. Funct. Anal. 150, No. 2, 498-525 (1997). ZBL0883.11017.

 Lauret, Emilio A., An asymptotic formula for representations of integers by indefinite Hermitian forms, Proc. Am. Math. Soc. 142, No. 1, 1-14 (2014). ZBL1329.11029.

 Gorodnik, Alexander; Nevo, Amos, The ergodic theory of lattice subgroups, Annals of Mathematics Studies 172. Princeton, NJ: Princeton University Press (ISBN 978-0-691-14185-5/pbk; 978-0-691-14184-8/hbk). 136 p. (2009). ZBL1186.37004.

 Gorodnik, Alexander; Nevo, Amos, Counting lattice points, J. Reine Angew. Math. 663, 127-176 (2012). ZBL1248.37011.

• In fact the work in your thesis on the consequences of Gorodnik and Nevo's result is almost exactly what I was hoping for when I asked this question. Indeed it is not completely explicit but it makes it clear what needs to be done. I see you have a link to your thesis on your personal web page. Would you consider adding a link to it here? I could then accept this as the answer. – Simon L Rydin Myerson Dec 6 '19 at 13:50
• One might also add that for the representation of integers by quadratic forms, there are many ways of obtaining such an asymptotic formula; the work of Heath-Brown (eudml.org/doc/153876) comes to mind. It looks like one difference in your work is that many of the intermediate steps apply to solutions in quaternionic integers, which as far as I know not true in other approaches. – Simon L Rydin Myerson Dec 6 '19 at 14:12
• I added a link to my web page where my Ph.D. thesis is available. I didn't use the link of the pdf since it was created by google drive and it may change (I hate broken links in MO). – emiliocba Dec 6 '19 at 17:13
• Yes, in principle this method should work for integers, integers of imaginary quadratic number fields, or integers associated to quaternion algebras (e.g. quaternionic integers). It may also have sense for octonionic integers (in case there exists such a thing). – emiliocba Dec 6 '19 at 17:18
• As explained in my update to the original post, I have accepted this as it gives a nontrivial but viable-looking strategy to answer the question. – Simon L Rydin Myerson Dec 8 '19 at 13:46