What's the relationship between Gauss sums and the normal distribution? Let $p$ be an odd prime and $\left( \frac{a}{p} \right)$ the Legendre symbol.  The Gauss sum
$\displaystyle g_p(a) = \sum_{k=0}^{p-1} \left( \frac{k}{p} \right) \zeta^{ak},$
where $\zeta_p = e^{ \frac{2\pi i}{p} }$, is a periodic function of period $p$ which is sometimes invoked in proofs of quadratic reciprocity.  As it turns out, $g_p(a) = \left( \frac{a}{p} \right) i^{ \frac{p-1}{2} } \sqrt{p}$, so $g_p(a)$ is essentially an eigenfunction of the discrete Fourier transform.  Now, if $(a, p) = 1$, we can write
$\displaystyle g_p(a) = \sum_{k=0}^{p-1} \zeta^{ak^2}$
so Gauss sums are some kind of finite analogue of the normal distribution $e^{-\pi x^2}$, which is itself well-known to be an eigenfunction of the Fourier transform on $\mathbb{R}$.  I remember someone claiming to me once that the two are closely related, but I haven't been able to track down a reference.  Does anyone know what the precise connection is?  Is there a theory of self-dual locally compact abelian groups somewhere out there?
 A: It's true (as the answer below and some of the commenters note) that it's easy to interpret this question in a way that makes it seem trivial and uninteresting.  I'm quite sure, however, that pursuing typographical similarity between $e^{x^2}$ and $\zeta^{m^2}$ leads to interesting mathematics, and so here's a more serious attempt at propoganda for some of Ivan Cherednik's work.
Pages 6,7,8 and 9 of Cherednik's paper "Double affine Hecke algebras and difference Fourier transforms" explain how to ``interpolate'' between integral formulas relating the Gaussian to the Gamma function and (a certain generalization of) Gauss sums.  
More explicitly, he shows that the formula (for many people, it's really just the definition of the Gamma function)
$$\int_{-\infty}^{\infty} e^{-x^2} x^{2k} dx=\Gamma \left( k+\frac{1}{2} \right)$$
(for $k \in \mathbb{C}$ with real part $>-1/2$) and the Gauss-Selberg sum
$$\sum_{j=0}^{N-2k} \zeta^{(k-j)^2/4} \frac{1-\zeta^{j+k}}{1-\zeta^k} \prod_{l=1}^j \frac{1-\zeta^{l+2k-1}}{1-\zeta^l}=\prod_{j=1}^k (1-\zeta^j)^{-1} \sum_{m=0}^{2N-1} \zeta^{m^2/4}$$
(where $N$ is a positive integer, $\zeta=e^{2\pi i/N}$ is a prim. $N$th root of $1$, and $k$ is a positive integer at most $N/2$) can both be obtained as limiting cases of the same $q$-series identity.  The common generalization of the Gaussian and the function $k \mapsto \zeta^{k^2}$ is the function $x \mapsto q^{x^2}$, and the measures weighting the integral and sum get replaced by Macdonald's measure---essentially the same one that shows up in the constant term conjecture for $A_1$, and that produces the Macdonald polynomials and kick-started the DAHA.  The Fourier transform is deformed along with everything else to produce the "Cherednik-Fourier" transform.
I don't know how much of the roots of unity story generalizes to higher rank root systems.
Note: In the Gauss-Selberg sum, replacing $k$ by the integer part of $N/2$ and manipulating a little (as in the nice exposition by David Speyer linked to in the question above) gives the usual formula for the Gauss sum.
A: I don't think there is anything deep going on here. The Fourier analysis on finite abelian group is fairly straightforward.
Gauss sums are the Fourier coefficients you get when you expand an additive character $k \rightarrow e^{\frac{2\pi iak}{p}}$ with respect to the basis of multiplicative characters (i.e. those that give rise to Dirichlet characters when our group is $\mathbb{Z}/n\mathbb{Z}$). A Gauss sum is a sum of the product of an additive and a multiplicative characters and as such can be thought of as a finite group analogue of the Gamma function. Recall that the Gamma function is the integral on $\mathbb{R}^{>0}$ of the product of $e^{-x}$ (additive character on the reals) and $x^s$ (a multiplicative character on $\mathbb{R}^{>0}$) with respect to the Haar measure $\frac{dx}{x}$ on $\mathbb{R}^{>0}$.
You are probably thinking ${\zeta_p}^{ak^2}$ as the finite analogue of the Gaussian $e^{-\pi x^2}$, but as you have written yourself,
$g_p(a)=\sum_{k=0}^{p-1} {\zeta_p}^{ak^2}$,
a Gauss sum is a sum of `things' that look like the Gaussian and there is no reason why a Gauss sum itself should be something like the Gaussian. 
