Fortunately, though, we don't need to posit a Set0 for this and end up with the same problem one turtle lower down. To formalise the fundamentals of proof theory, we just need to be able to talk about manipulating strings of symbols, so a theory of the natural numbers (eg PA or even HA) is more than enough. On the other hand, we do have to presume some given meta-theory (as most traditional logicians would call it) or logical framework (as many computer scientists would) to get off the ground, and for that we really do have the problem that we can't talk about it as an object itself without passing to a meta-meta-theory. This is a real problem, but more a philosophical than a mathematical one: we just have to either accept a potential infinite regress, or make a leap of faith that facts proven within our meta-theory, about some internal version of it, will apply to the meta-theory itself (whether this is a platonic object, a physical computer system, or whatever else).
The second issue is one of reflection, and has a more satisfying resolution. (I'll keep the meta-theory informal, but PA or even HA would be more than enough to formalise this, I think.) Say we have some axioms for Set1, strong enough that it contains an "internal copy" of the natural numbers and hence can talk about basic proof theory; then define Set2 as the "internal version" of the same theory in Set1, and so on. Now we can prove:
Suppose Set1 proves Con(Set2). By the above proposition, Set1 also proves "Set2 proves Con(Set3)". Now, by Gödel'sGödel's theorem for Set2, we can deduce (still in Set1) the theorem "Set2 is inconsistent**"inconsistent". But now we have proofs in Set1 of both Con(Set2) and its negation; so Set1 is inconsistent. QED