If the Riemann Hypothesis fails, must it fail infinitely often? That is must there either be no non-trivial zeros off the critical line or
infinitely many?
I'm sure that no one believes otherwise, but I've never seen a theorem in the
literature addressing this.   Folklore perhaps?
 A: This does not answer your question, but recall that the explicit formula reads
$$\psi(x) = x - \sum_{\rho} \frac{x^{\rho}}{\rho} - \log(2 \pi)  - \log(1 - x^{-2})/2.$$
On RH, this implies that $\psi(x) = x + O(x^{1/2} \log^2 x)$. If RH were false with infinitely many exceptions, you'd get a bigger error term which would potentially behave erratically and "randomly". But if RH were false with finitely many exceptions, then you would get
$$\psi(x) = x - \sum_{\rho} \frac{x^{\rho}}{\rho} + O(x^{1/2} \log^2 x),$$
with the sum over the exceptions -- a distinct secondary term and therefore weird oscillatory behavior in the frequency of the primes. Although this has not been ruled out to my knowledge (for example you see terms for Siegel zeroes of Dirichlet $L$-functions all over the literature), it does seem unlikely.
A: To elaborate on my comment, I really do think this won't be resolved until RH itself is resolved, or at least until new tools are brought onto the scene, because it's so far from what we know about the zeros by current technology.  All the theorems we have about zeros of the zeta function - e.g., the zero-free region, the zero-density theorems, Montgomery's conditional results on pair correlation, Selberg's theorems on $S(t)$ - are either incredibly weak compared to what we expect to be true (Iwaniec often refers to zero-density theorems as "estimates for the cardinality of the empty set"), or are theorems "in the large", that is to say they deal in whole masses of zeros as opposed to individual zeros.  Conventional harmonic analysis is simply unable to grasp individual zeros, for reasons of the uncertainty principle.
The closest reference I know to your question is a paper of Bombieri, "Remarks on Weil's quadratic functional in the theory of prime numbers", where he shows that if the RH is false for only a finite number of zeros, then very odd things follow... 
Edit: Thinking about this question a bit more has led me to a conjecture seems quite a bit weaker than anything approaching RH and which essentially implies the impossibility of finitely many failures.
A: To quote from "Problems of the Millennium: the Riemann Hypothesis" by Bombieri:

It is known that hypothetical exceptions to the Riemann hypothesis must be rare if we move away from the line $\Re(s) = \frac{1}{2}$.
Let $N(\alpha, T)$ be the number of zeros of $\zeta(s)$ in the rectangle $\alpha \leq \Re(s) \leq 2$, $0 \leq \Im(s) \leq T$. The prototype result goes back to Bohr and Landau in 1914, namely $N(\alpha, T) = O(T)$ for any fixed $\alpha$ with $\frac{1}{2} < \alpha < 1$. A significant improvement of the result of Bohr and Landau was obtained by Carlson in 1920, obtaining the density
theorem $N(\alpha, T) = O(T^{4\alpha(1−\alpha)+\epsilon})$ for any fixed $\epsilon > 0$. The fact that the exponent here is strictly less than $1$ is important for arithmetic applications, for example in the study of primes in short intervals. The exponent in Carlson’s theorem has gone through several successive refinements for various ranges of $\alpha$, in particular in the range $\frac{3}{4} < \alpha < 1$. Curiously enough, the best exponent known up to date in the range $\frac{1}{2} < \alpha \leq \frac{3}{4}$ remains Ingham’s exponent $3(1 − \alpha)/(2 − \alpha)$, obtained in 1940.

A: On a somewhat related note I would like to call attention to the following papers:
A. Booker, Poles of Artin $L$-functions and the strong Artin conjecture, Annals of Math. 158 (2003), 1089-1098.
Here the author proves that for 2-dimensional Galois representations $\rho$ the Artin conjecture implies the strong Artin conjecture by showing that if some character twist $L(s,\rho\otimes\chi)$ has a pole then $L(s,\rho)$ has infinitely many poles (hence in fact all twists have infinitely many poles).
P. Sarnak, A. Zaharescu, Some remarks on Landau-Siegel zeros, Duke Math. J. 111 (2002), 495-507.
Here the authors prove that if all zeros of all quadratic Dirichlet $L$-functions are on the critical line or on the real axis, then the possible real zeros are much farther from $s=1$ than we can prove at present without any hypothesis.
