That is must there either be no nontrivial zeros off the critical line or infinitely many?
I'm sure that no one believes otherwise, but I've never seen a theorem in the literature addressing this. Folklore perhaps?
That is must there either be no nontrivial zeros off the critical line or infinitely many? I'm sure that no one believes otherwise, but I've never seen a theorem in the literature addressing this. Folklore perhaps? 


This does not answer your question, but recall that the explicit formula reads $$\psi(x) = x  \sum_{\rho} \frac{x^{\rho}}{\rho}  \log(2 \pi)  \log(1  x^{2})/2.$$ On RH, this implies that $\psi(x) = x + O(x^{1/2} \log^2 x)$. If RH were false with infinitely many exceptions, you'd get a bigger error term which would potentially behave erratically and "randomly". But if RH were false with finitely many exceptions, then you would get $$\psi(x) = x  \sum_{\rho} \frac{x^{\rho}}{\rho} + O(x^{1/2} \log^2 x),$$ with the sum over the exceptions  a distinct secondary term and therefore weird oscillatory behavior in the frequency of the primes. Although this has not been ruled out to my knowledge (for example you see terms for Siegel zeroes of Dirichlet $L$functions all over the literature), it does seem unlikely. 


To elaborate on my comment, I really do think this won't be resolved until RH itself is resolved, or at least until new tools are brought onto the scene, because it's so far from what we know about the zeros by current technology. All the theorems we have about zeros of the zeta function  e.g., the zerofree region, the zerodensity theorems, Montgomery's conditional results on pair correlation, Selberg's theorems on $S(t)$  are either incredibly weak compared to what we expect to be true (Iwaniec often refers to zerodensity theorems as "estimates for the cardinality of the empty set"), or are theorems "in the large", that is to say they deal in whole masses of zeros as opposed to individual zeros. Conventional harmonic analysis is simply unable to grasp individual zeros, for reasons of the uncertainty principle. The closest reference I know to your question is a paper of Bombieri, "Remarks on Weil's quadratic functional in the theory of prime numbers", where he shows that if the RH is false for only a finite number of zeros, then very odd things follow... Edit: Thinking about this question a bit more has led me to a conjecture seems quite a bit weaker than anything approaching RH and which essentially implies the impossibility of finitely many failures. 


To quote from "Problems of the Millennium: the Riemann Hypothesis" by Bombieri:



On a somewhat related note I would like to call attention to the following papers: A. Booker, Poles of Artin $L$functions and the strong Artin conjecture, Annals of Math. 158 (2003), 10891098. Here the author proves that for 2dimensional Galois representations $\rho$ the Artin conjecture implies the strong Artin conjecture by showing that if some character twist $L(s,\rho\otimes\chi)$ has a pole then $L(s,\rho)$ has infinitely many poles (hence in fact all twists have infinitely many poles). P. Sarnak, A. Zaharescu, Some remarks on LandauSiegel zeros, Duke Math. J. 111 (2002), 495507. Here the authors prove that if all zeros of all quadratic Dirichlet $L$functions are on the critical line or on the real axis, then the possible real zeros are much farther from $s=1$ than we can prove at present without any hypothesis. 

