There is a famous circular argument for the Prime Number Theorem (PNT). It turns out that there exists an infinite sequence of elementary-to-prove Chebyshev-type estimates that taken together imply PNT. Unfortunately, the collective existence of all these proofs seems to require the PNT, so one must work hard a la Selberg and Erdos for an elementary proof. See Harold Diamond, Elementary methods in the study of the distribution of prime numbers, Bull. Amer. Math. Soc. N. S. 7 (1982), 553-589.

Now on the traditional view, circular proofs simply have no value at all. Yet one feels perhaps that the example in the previous paragraph has something to say. Just an illusion? Or does there exist a foundational framework where circular proofs of this special sort enjoy bona fide status?

Just to riff a little more, imagine that the Goldbach conjecture turns out independent of PA (or some other, perhaps weaker, axiom system for arithmetic). The truth of the Goldbach conjecture (required for independence!) would then imply the existence of a proof (trivial!) for any given even number that that one even number equals the sum of two primes. Now that obviously isn't very interesting compared to the example in the first paragraph, and perhaps the difference has something to do with the greater quantifier complexity of PNT?

As a side question, are there any similar stories in the lore of the Riemann Hypothesis? For example, does RH imply the existence of an infinite sequence of zero-free region proofs of a known form that collectively amount to RH?