Can Gauss sums derandomize any heuristic arguments? I've always been fascinated by the fact that the classical Gauss sum has absolute value $\sqrt p$, which is exactly what we would expect if we were to interpret the Gauss sum as a random walk.  In particular, I have long wondered whether this apparent link between number theory and probability theory is just a curiosity, or whether it is a symptom of a deeper connection.  For example, there are many heuristic arguments in number theory that treat primes as being "random."  Is there any example of such a heuristic argument that can be made rigorous by using the above observation about Gauss sums?
 A: The thing is that it is well known that for the quadratic Gauss sums, expressed as an exponential sum rather than with Legendre symbols, the path is very much not a random walk when you plot it in the complex plane. There is plenty of structure visible as approximate Cornu spirals. 
Such sums are not the only Gauss sums, as I know to my cost; and the quadratic case is atypical (perhaps). But from a high-flown point of view, a Gauss sum is a special function in the theory of finite fields (like a Gamma function). There seems to be more mileage in asking about what is special about it.
A: Caveat lector: as an answer to Timothy's question, this is tangential at best.
Regarding Gerry and Kevin's comments, people might be interested in Section 2 of these notes of mine
from an undergraduate number theory course, in which the philosophy of "almost square root error" is batted around for a while, especially with regard to the Riemann hypothesis.  This is maybe my favorite set of lecture notes from this course, perhaps because I got a chance to (talk and) write excitedly about things I hardly understand: I certainly do not claim a professional level of insight here.  (Indeed some regulars on this site could do far better, and I would be interested to hear their critiques.)
The upshot though is that of agreement with Gerry and Kevin: having size $\sqrt{p}$ on the nose is just a little bit better than random, but the little bit makes a big difference: to me this suggests very precise structure rather than randomness.
Please see here for another take on the subject of almost square root error in the context of the Sato-Tate Conjecture.  This latter article was written at almost the same time as mine, and the author happens to be my former thesis advisor.  I am reasonably sure that both of these coincidences are indeed coincidental.
A: Actually I would think there are connections. Even if the the coincidence $\sqrt{p}$ seems ordinary, there are low-correlation sequences which owe their low correlation to Gauss sum estimates and looking from a probabilistic view point sequences, I would think there would be interpretations from a view point of uncorrelated random variables. Your starting point could be low-correlation sequences used in communications systems and beyond that I would think Nicholas Katz and Sarnak's dive into random matrices would help.
http://books.google.com/books?id=wXyOPbzvowsC&printsec=frontcover&dq=inauthor:%22Nicholas+M.+Katz%22&hl=en&ei=j9IrTu24CpKnsAKG5c3DCw&sa=X&oi=book_result&ct=result&resnum=5&ved=0CD8Q6AEwBA#v=onepage&q&f=false
