For what it's worth, the reason I made that comment was because when I gave talks expressing skepticism about the philosophical basis of ZFC, I would often get the reaction "but as long as it's consistent, what's the problem?" Some version of this attitude is also quite prevalent in the philosophical literature on the subject. I wanted to make the point that just being consistent isn't good enough: ZFC might well be consistent yet still prove, for example, that some Turing machine halts when in fact it does not. If you're a skeptic about sets but not about numbers, this could be a concern.
As for the "safety" of assuming ZFC is arithmetically sound, it is clear that as long as ZFC is consistent, no weaker system could prove that it is unsound. So for predicativists like me, who generally work in much weaker systems than ZFC, it seems unlikely that we could ever establish unsoundness. In that sense I'd say it's pretty "safe" to assume it is sound.
However ... I regard the Peano axioms as expressing intuitively evident truths about the natural numbers, so I have no problem affirming Con(PA). Whereas I regard ZFC as what you get when you realize that full comprehension leads to inconsistencies, so you replace it with a hodgepodge of instances of comprehension which appear to be weak enough to block any paradoxes. It seems to me quite ad hoc and unmotivated, so although we might (or might not) be confident that ZFC is consistent, there is little reason to expect it to be (or not to be) arithmetically sound.
(I should add that I regard the "iterative conception" which is supposed to justify ZFC as utterly unconvincing. Sets are said to be built up in stages, and then it is parenthetically added that they are, of course, timeless abstract objects so they aren't really "built", nor do they really "appear" in "stages" --- it's all just a "metaphor". Thus "any conviction that the iterative conception may carry is made to depend on metaphorical details that are dismissed as inessential to it." (I. Jane))