George Boolos, eminent philosopher and logician, wrote as follows (perhaps slightly tongue-in-cheek):

He continued for a bit, and then...

(From G. Boolos, Logic, Logic, and Logic, Chapter 8: "Must we Believe in Set Theory?")

So, there you are: the answer to your question is "yes". At least he suggests that perhaps "set theory is not true" (not the same as "formally inconsistent"). The cardinal he mentions is not normally regarded as "large": it isn't even weakly inaccessible. The proof that it exists is easy.

My personal knowledge of formalising mathematics in the proof assistant Isabelle/HOL suggests that ZFC is much, much stronger than necessary for most mathematics with the obvious exception of the study of ZFC itself. A great body of advanced mathematical constructions – even Grothendieck schemes, thought by some to require more than ZFC – went easily into Isabelle/HOL. Higher-order logic is weaker even than Zermelo set theory, which (lacking the axiom of Replacement) is itself much weaker than ZFC. So if ZFC were somehow found to be inconsistent, mathematics would survive.