Why does the Riemann zeta function have non-trivial zeros? This is a very basic question of course, and exposes my serious ignorance of analytic number theory, but what I am looking for is a good intuitive explanation rather than a formal proof (though a sufficiently short formal proof could count as an intuitive explanation). So, for instance, a proof that estimated a contour integral and thereby showed that the number of zeros inside the contour was greater than zero would not count as a reason. A brief glance at Wikipedia suggests that the Hadamard product formula could give a proof: if there are no non-trivial zeros then you get a suspiciously nice formula for ζ(s) itself. But that would feel to me like formal magic. A better bet would probably be Riemann's explicit formula, but that seems to require one to know something about the distribution of primes. Perhaps a combination of the explicit formula and the functional equation would do the trick, but that again leaves me feeling as though something magic has happened. Perhaps magic is needed.
A very closely related question is this. Does the existence of non-trivial zeros on the critical strip imply anything about the distribution of prime numbers? I know that it implies that the partial sums of the Möbius and Liouville functions cannot grow too slowly, and it's really this that I want to understand. 
 A: If the Riemann zeta function had only trivial zeroes, then after multiplying by the gamma factor, it would become a zero-free entire function of finite order. Every zero-free entire function of finite order is of the form $\exp(poly(z))$. This cannot happen for the zeta function, as can be seen by considering behavior as $s\to\infty$ along the reals, for example.
A: Here's a variation on the arguments using the explicit formula.  If there were no zeros in the critical strip then the explicit formula would say $$\psi_0(x) = x - \frac{\zeta'(0)}{\zeta(0)} - \frac12 \log(1-x^{-2}),$$ where $\psi_0(x)$ is the same as the usual $\psi(x)$ except when $x$ takes integer values $\psi_0(x)$ is the average of the left and right limits of $\psi(x)$.  The right hand side is continuous for $x >0$ while the left hand side is not.  The argument generalizes to show that there cannot be finitely many zeros in the critical strip, or even an infinitude that do not grow "too quickly."  Precisely, if $$\sum_{\rho} \frac{x^{\rho}}{\rho}$$ converged fast enough to form a continuous function of $x$ then the same argument would carry over.
A: Rather than the problem of why the zeta function has non-trivial zeros, let me address 
 Gowers's question of why the error term in the prime number theorem needs to be large.   The short answer that I propose is: because the integers are so well distributed. To make this precise, I shall prove a general result on semigroups, showing that 
 either the "integers" in the semigroup or the "primes" must be poorly distributed -- one may think of this as an "uncertainty principle" that both primes and integers cannot be simultaneously smoothly behaved. This has the flavor of  Beurling generalized primes, but I don't recall this result in the literature; maybe it exists already (indeed it does, see the edit below). Also note that the proof will not make any use 
 of the functional equation for $\zeta(s)$ as this does not exist in a general semigroup. 
EDIT:  Indeed doing a literature search a few hours after posting this, 
I found a paper of Hilberdink http://www.sciencedirect.com/science/article/pii/S0022314X04002069 which proves the 
Theorem below, and with a similar method of proof. 
Suppose that $1<p_1< p_2 <\ldots$ is a sequence of real numbers (the "primes"), and 
 $1=a_1 \le a_2 \le \ldots$ is the semigroup generated by them (the "integers").  Initially I made the assumption that the "integers" are distinct, and satisfy a mild spacing 
condition $a_{n+1}-a_n \gg n^{-1}$ (so that in particular there is unique factorization), but this is not necessary. Let $N(x)$ denote the number of 
"integers" below $x$, and 
$$
P(x) = \sum_{p_n^{k} \le x} \log p_n,
$$ 
which is the analog of the usual $\psi(x)$ (counting prime powers with weight $\log p$).  Assume that for some $\delta>0$ 
$$
N(x) = Ax +O(x^{\frac 12-\delta}), 
$$ 
for some non-zero constant $A$, and that 
$$ 
P(x) = x + O(x^{\frac 12-\delta}). 
$$ 
Theorem.   Either the asymptotic formula for $N(x)$ or the asymptotic 
formula for $P(x)$ must fail. 
Put 
$$ 
\zeta_A(s) = \sum_{n=1}^{\infty} a_n^{-s} = \prod_{n} \Big(1-\frac{1}{p_n^s}\Big)^{-1}. 
$$
By our assumptions on $N$ and $P$, the sum and product above converge absolutely in Re$(s)>1$. By the assumption on $N(x)$, $\zeta_A(s)$ extends to an analytic function in Re$(s)>1/2-\delta$ except for a simple pole at $s=1$ with residue $A$.  By the assumption on $P$, we see that the logarithmic derivative 
$$ 
-\frac{\zeta_A^{\prime}}{\zeta_A}(s) = \sum_{n, k} \frac{\log p_n}{p_n^{ks}} 
$$ 
extends analytically to Re$(s)>1/2-\delta$ except for a simple pole at $1$.  Thus 
$\zeta_A(s)$ has no zeros in Re$(s)>1/2-\delta$.  
New edit: Sketch of a second proof.  Adapting the argument that the Riemann hypothesis implies the Lindelof hypothesis (see below), we obtain that $|\zeta_A(s)| \ll (1+|s|)^{\epsilon}$ provided $s$ is not close to the pole at $1$, and that Re$(s)>1/2-\delta/2$.  From this and a standard contour shift argument we find that for large $N$ and any $t$, 
$$ 
\sum_{a_n\le N} a_n^{it} = \frac{N^{1+it}}{1+it} +O(N^{1/2-\delta+\epsilon} (1+|t|)^{\epsilon}). 
$$
What is used here is that we have Lindelof even a little to the left of the half line.  
But the above identity can be seen to contradict the Plancherel formula.  More precisely, let $T$ be a large power of $N$, and let $\Phi$ be a non-negative function supported in $[-1,1]$ with non-negative Fourier transform.  Then we see that (discarding all but the diagonal terms)
$$ 
\int_{-\infty}^{\infty} \Big| \sum_{a_n\le N} a_n^{it}\Big|^2 \Phi(t/T) dt 
\ge T{\hat \Phi}(0) \sum_{a_n\le N} 1 \sim TN {\hat \Phi}(0).
$$ 
On the other hand, if we use our identity then the above is seen to be 
$$ 
\ll N^2 + T^{1+\epsilon} N^{1-2\delta}. 
$$ 
This is a contradiction.
Original Proof:  Below let's assume always that we are in the region Re$(s)>1/2-\delta$, and that 
the imaginary part is large so that we are not near the pole at $1$.  From 
the analytic continuation of $\zeta_A$ (using that $N(x)$ is very regular), it follows 
that there is an a priori polynomial bound $|\zeta_A(s)|\ll |s|^{B}$ in the region Re$(s)>1/2-\delta/2$.  Thus there is a bound for the real part of $\log \zeta_{A}(s)$, and by the Borel-Caratheodory lemma (standard complex analysis) one can bootstrap this to a bound for $|\log \zeta_A(s)|$.  Then applying the Hadamard three circle theorem to $\log \zeta_A(s)$ one obtains a much better bound: $|\zeta_A(s)| \ll |s|^{\epsilon}$.   This is the usual proof that Riemann implies Lindelof.  (At this stage, if we knew a "functional equation" we'd be done, as the usual $\zeta(s)$ is large when Re$(s)<1/2$.  This point appeared in Matt Young's answers earlier.)
Knowing the Lindelof hypothesis for $\zeta_A(s)$ in Re$(s)> 1/2-\delta/2$, we can 
show the following approximate formula: for $\sigma > 1/2- \delta/4$ 
$$ 
\zeta_A(\sigma +it) = \sum_{a_n \le N} a_n^{-\sigma-it} + O(|t|^{\epsilon} N^{-\delta/4}). 
$$ 
The proof is standard;  see the penultimate chapter of Titchmarsh for the real $\zeta(s)$ where this holds when $\sigma$ is strictly bigger than $1/2$, and our stronger result is true because we have Lindelof in a wider region.  
Now we are ready to get our contradiction.  Consider for large $T$ 
$$ 
\int_T^{2T} |\zeta_A(1/2+it)|^2  dt. 
$$ 
To do this carefully it may be helpful to put in a smooth weight $\Phi(t/T)$ above 
(but this is not a paper!).  Using the approximate formula derived above, and our mild spacing condition $a_{n+1}-a_n \gg n^{-1}$, 
we may see that for any $T^{\epsilon} \le N \le T^{1/10}$ we have 
$$ 
\int_T^{2T} |\zeta_A(1/2+it)|^2 dt \sim T \sum_{a_n \le N} a_n^{-1} \sim AT \log N.
$$ 
But that's absurd! This completes our sketch proof. 
A: An intuitive reason is indicated by Bukh in another answer.  The reason there are nontrivial zeros is because the zeta-function is known to grow in a way that it wouldn't grow if there were no nontrivial zeros. 
Here is some additional (but not complete) detail. Let's pass from the zeta-function to the completed zeta-function 
$$
Z(s) = \pi^{-s/2}\Gamma\left(\frac{s}{2}\right)\zeta(s).
$$
The functional equation for the zeta-function is equivalent to the equation $Z(1-s)=Z(s)$.
What does this tell us?  By the Euler product, $\zeta(s)$ is nonvanishing in the half-plane to the right of 1.  The $\pi$ and $\Gamma$ factors in $Z(s)$ are also nonvanishing in that half-plane, so $Z(s)$ is nonzero if the real part of $s$ is greater than 1.  From the equation $Z(1-s) = Z(s)$, $Z(s)$ is also nonvanishing if the real part of $s$ is less than 0.  Therefore the nontrivial zeros of $\zeta(s)$ are the same thing as all zeros of $Z(s)$. 
Thus the question of why $\zeta(s)$ has nontrivial zeros is the same as the question of why $Z(s)$ has any zeros at all (nontrivial zeta zeros = zeros of $Z(s)$). 
There are (simple) poles for $Z(s)$, at $s = 0$ and $s = 1$.  So consider 
the function $F(s) = s(s-1)Z(s)$.  This is an entire function which satisfies $F(1-s) = F(s)$. Let $|F|_r$ be the maximum of $|F(s)|$ on the circle around the origin of radius $r$. Using analytic methods and the functional equation (to transfer information from right to left half-planes), $|F(s)|_r$ grows exponentially with $r$ and, more precisely,  $\log|F|_r$ is asymptotic to $(1/2)r\log r$. From this growth estimate and the Hadamard product formula, if $F(s)$ had only finitely many zeros 
then $F(s) = e^{As}P(s)$ for some constant $A$ and some polynomial $P(s)$.  The functional equation for $F(s)$ then forces $A = 0$, so $F(s)$ is a polynomial.  But polynomial functions don't have their max. modulus on circles grow exponentially.  So we have a contradiction, which forces $F(s)$ to have infinitely many zeros and thus 
$\zeta(s)$ has infinitely many nontrivial zeros.
This may be looking too technical, but really the basic point is similar to the proof by Liouville's theorem of the fundamental theorem of algebra: if $p(z)$ were a nonconstant polynomial without zeros in the complex plane then we'd run into an inconsistency comparing the known behavior of $|1/p(z)|$ (continuous and tending to 0 as $|z| \rightarrow \infty$) with what would follow if there were no zeros ($1/p(z)$ is entire and nonconstant).  Is that explanation of why $p(z)$ must have a zero intuitive or is it formal magic? We can't expect a formula for a zero of $p(z)$, so in some sense this growth argument with $|1/p(z)|$ to conjure up a zero indirectly is magical.  But it has a simple elementary character to it as well:  there has to be a zero because otherwise $1/p(z)$ would grow in a way inconsistent with how we know for sure it grows.  In the same way, $\zeta(s)$ has to have infinitely many (not only a few) nontrivial zeros because otherwise $\zeta(s)$ would grow in a way that is inconsistent with how we know it grows. (I am glossing  over the distinction between $\zeta(s)$ and $Z(s)$ and $s(s-1)Z(s)$ since we're supposed to be putting our intuitive hats on.)
A: (I've just typed a comment but somehow it got lost. I'll type an answer instead).
Bloom is right, no zeros implies too nice a formula, except the right approximation to $\pi(x)$ is $\int_2^x dt/\log t$, not $x / \log x$. The existence of zeros implies that the error cannot be better than $O(x^{1/2})$.
On the other hand, the zeta function of ${\mathbb{F}}_q[t]$ has no zeros (and thus there is a nice formula for the analogue of $\pi(x)$) so there is no philosophical reason for a zeta function to have zeros. You might interpret the existence of infinitely many zeros of the zeta function of $\mathbb{Z}$ to mean that $Spec {\mathbb{Z}}$ has "infinite genus".
A: May I offer an attempt at a very lowbrow answer?  This answer might naturally strike anyone who knows the functional equation, who looks at a color coded graph of zeta function such as http://en.wikipedia.org/wiki/File:Complex_zeta.jpg, and who knows a little about the long-range behavior of exponential functions and $\Gamma(z)$.  Nothing I will say is beyond the ken of an average serious undergraduate.
$\zeta(s)\rightarrow 1$ as $\Re(s)\rightarrow +\infty$, so no zeros occur far into the right half-plane (the Euler product makes this clear too, indeed no zeros to the right of the critial strip).  Then the functional equation prevents also non-trivial zeros far into the left half-plane (indeed to the left of the critical strip).   
Now the argument of the function oscillates as you move up vertical lines far into the left half-plane, as one learns from the factors of the functional equation, and their long-range behavior and the right-plane limiting behavior of $\zeta(s)$ itself.
So think about the curves that constitute the real locus and the purely imaginary locus of $\zeta(s)$.  These curves can't cross anywhere except on the negative real axis and in the critical strip.   Some of these curves do cross on the negative real axis, at the trivial zeros.  But there is a rightmost trivial zero, it has its two curves, and there still remain infinitely many other curves lying above those.
No non-trivial zeros would mean no crossing so no doubling back for all these infinitely many curves.  What else could they do?  They could only head up into the critical strip, packing tightly together.  That would imply faster than exponential decay along some locus as you moved up the strip.  And why is that impossible?
[I see now that my attempt at a punchline was based on an unjustified assumption.  I'm still thinking about a valid and simple replacement.  Suggestions welcome.]
