Edit: Let me now put forth a more complete answer tying together philosophical and mathematical considerations. A strict finitist doesn't believe in the existence of the set of Natural numbers but that in itself does not prevent the individual from talking about properties shared by all of its elements. Even if you don't believe that it is a static collection of elements that can be put together into a single set, you can still syntactically prove theorems about Natural numbers (as discussed in more detail by Carl with reference to ZFC). But since $\mathbb{N}$ does not exist in this platonistic frame of thinking, your intuitive objection about quantifying over all well-founded sets is analogous to this problem where we try to quantify over all Natural numbers. Ultimately, some finitists will adopt a compromise where they accept the existence of infinite sets but only ones that can be constructed from the Natural numbers in a finitistic manner (e.g., expressible in a relatively weak theory such as primitive recursive arithmetic or PRA). A somewhat analogous resolution to your philosophical objection is to allow for the existence of definable classes as Kunen does by considering them as "abbreviations for expressions not involving them" or to treat classes as separate formal objects in a conservative extension of ZFC such as NBG. Of course, if you assume the existence of an inaccessible cardinal $\kappa$, then you have a nice set model of NBG, mainly the collection of $\Delta_0$-definable subsets of $V_{\kappa}$.
Finally, you can throw yourself in a constructed set model $M$ guaranteed by ZFC's believed consistency and restrict to some subclass of $M$. Specifically, you can externally take a sufficient cut of its universe $V_{\alpha}^M$ (which may be all of $M$) as you alluded to or restrict yourself to its parameter-free definable sets if it happens to be a model of $V = OD$ as François mentions to get a model of ZFC. However, in both cases, you will potentially be sacrificing some of the richness exhibited by the universe $M$.