People often reason like the OP, but I think the argument is invalid. Suppose a theory proves 2+2=4. According to the OP's reasoning, if the system is inconsistent, it proves everything, including 2+2=4. Therefore, you haven't proven anything, because maybe the system is inconsistent. But you have proven something! You have proven 2+2=4. (Taking Joel's analogy in comments after the question, you can't believe what a con man says about anything, not just a recommendation for himself.)

I thiink the OP has in the back of his mind a rather standard notion of mathematical proof - one starts with axioms one believes are true, and then by proving things, one establishes that other assertions called theorems are true, but these theorems can have no more epistemological certainty than the axioms. So proving the consistency of a theory, within a theory, does not add any epistemological certainty about its consistency, because we already know the theory is consistent - after all, the axioms are true, and truths can only validly prove truths.

But actually, you don't just know that a theory is consistent, just because its axioms are true. Think of it - maybe there are true axioms which produce an inconsistent theory. You start naively churning out theorems from the axioms and - boom - unexpectedly out comes the contradiction. Sure, there is an argument that says this can't be so - true axioms only validly prove truths, and a contradiction is not true - but that is just saying we can provide a proof. And that is what a theory is supposed to do - provide proofs.

That is, you believe in certain axioms X to be true. You want to know whether or not the theory T beginning with these axioms X is consistent. It might not be, who knows? Maybe T proves everything. You therefore set about trying to prove T's consistency, perhaps by formalizing the argument already cited. If you are able to prove it using only axioms which you believe to be true, then you can have confidence that T is consistent. Therefore, if you can prove it with the axioms X, you can have this confidence. Maybe you can prove it with other axioms in which you believe, but since you already said you thought axioms X to be true, using them and only them is kinda a nice feature.

Now someone will surely point out that, just because you have proved T is consistent, doesn't give you, in some sense, greater grounds to believe in the axioms X. And that is completely correct. But that wasn't the question. That is, proving the consistency of a system within itself can give an a priori reason to believe in the consistency of the system - it will if the axioms can be known a priori, and the rules of inference can be seen as valid a priori. What it cannot do is give an a priori reason to believe in the axioms of the system. (It may give a reason, but that reason is not a priori.)