I read somewhere that Riemann believed he could find a representation of the zeta function that would allow him to show that all the nontrivial zeros of the zeta function lie on the critical line. I am wondering, then, is there any record of his attempts to prove RH?
The short answer is no. If anyone were aware of such a record, it would surely have been Carl Siegel, who undertook a careful study of Riemann’s unpublished notes. However, Siegel wrote:
Approaches to a proof of the socalled “Riemann hypothesis” or even to a proof of the existence of infinitely many zeros of the zeta function on the critical line are not included in Riemann’s papers.
Riemann himself, in his paper on the zeta function, said only that he made “some fleeting, vain attempts” (einigen flüchtigen vergeblichen Versuchen) to prove (what we now call) the Riemann hypothesis, and gave no indication that he recorded these attempts.
Having said that, I want to mention that there is some interesting speculation in Chapter 7 (on the Riemann–Siegel formula; see especially section 7.8) in H. M. Edwards's book Riemann’s Zeta Function about what Riemann’s train of thought might have been.

$\begingroup$ I am familiar with Riemann's paper and Edward's book. Maybe I should have asked if anyone has extensively studied Riemann's Nachlass in Gottingen to see if it contains any of Riemann's efforts on trying to prove RH. His computations on the first few nontrivial zeros of zeta are there. I don't believe that Riemann underestimated his conjecture because of the connection of the zeros with the primes via the explicit formulae. $\endgroup$ – Mustafa Said Jan 2 at 4:53

5$\begingroup$ @MustafaSaid : Siegel doesn't count as having extensively studied Riemann's Nachlass? $\endgroup$ – Timothy Chow Jan 2 at 5:03

$\begingroup$ Thank you for pointing this out as I am not very familiar with Siegal investigation of zeta. $\endgroup$ – Mustafa Said Jan 2 at 5:08