My answer is more of an extended comment on what Tim has already said. I guess what is at stake here is precisely this question: what does it mean the expression $CON(PA)$?
The common consensus is that it is a metamathematical statement, and indeed it is.
But what is metamathematics? Answer: other mathematics. In principle there is no distinction between Proof Theory of Peano Arithmetics, and , say, the Theory of Sobolev Spaces. In proof theory one studies structures, such as proof trees, which, in their intended meaning, talk about formal proofs of an underlying math theory, in this case Peano Arithmetics. But here is the deal: thanks to a special encoding, one can express certain statements of the "meta"-theory in the base theory, and so, in the case of PA, models of such a theory have some arithmetical statements which code some metamathematical facts on the theory. One of these is the infamous CON(PA).
Now, let us imagine the category of countable models of PA, with corresponding maps. That is our "Arithmetical Multiverse". The chief point to unravel the seeming paradox raised by the PO is that PA does not know anything about actual infinity.
All models, bar none, "think" that they are made of standard numbers.
It so happens that, "from outside" (ie from the perspective of an underlying set theoretical universe), the category above has a distinguished object, namely an initial object. We call that initial object $N$. The metamathematics encoded in $N$ is the "true" metamathematics, and everything else is (again, that is common folklore), gibberish.
But let us take a slightly different look at the story:
let us assume for a moment that all models of PA are in some sense equal. Each has its own coded meta-theory. In some of them, NON-CON(PA) is true. If this statement happened to be true in $N$ it would be an earthquake, because in that case there would be a finite term witnessing a proof of inconsistency. But (assuming the consistency of PA), that is not the case. Note that, although N is an initial segment of all of the arithmetical universes, none has any idea about it, they have no capability to define it with a first-order formula. Everything looks to them just some standard arithmetics.
Now, let us play this game: to every model of PA let us assign a radius, the Radius of Consistency: it is measured by an element in the model, the minimal element (when it is there!) that proves the inconsistency of PA.
If that element does not exist, we say that the radius for that model is unbounded. Note en passant: as all models have the same order type, namely ω + (ω* + ω) ⋅ η, we can actually define the radius of consistency by its position in this order. I shall not pursue this topic further here, for the sake of brevity.
Armed with the Radius of Consistency, we can say that the conventional consistency of PA is the meta-statement that N 's radius is unbounded.
Similarly, there are models of PA who have a much larger radius. Those are exactly the models which are conjured up by the PO.
You may say: all good and well, but these statements are still gibberish to me. Perhaps they encode garbage, useless information, as far as metamathematics goes (obviously, their are legitimate statements from the point of view of algebra).
Perhaps, but not so fast. For instance, sub-theories of PA such as $I\Sigma_1$ are finitely axiomatizable, so for those arithmetical multiverses the axioms involved in the "gibberish "are in fact real axioms. And things may get even odder. I think it is fair to say that so far nobody has done a detailed analysis of these statements of inconsistencies in complete details. There may be many surprises there. But here I stop...