Here are two attempts to answer what I view is the underlying question. The first attempt is taken from "Some Philosophical Prolegomena" (a section of the notes "The Axioms of Set Theory", by Tom Foster).
Many people come to set theory having been sold a story about its foundational significance; such people are often worried by apparent circularities such as...
Before we even reach set theory we have to have the language of first-order logic. Now the language of first-order logic is an inductively defined set and as such is the minimal [with respect to inclusion] set satisfying certain closure properties, and wasn’t it in order to clarify things like this (among others) that we needed set theory . . . ? And how can we talk about arities if we don’t already have arithmetic? And weren’t we supposed to get arithmetic from set theory?
...this doesn’t mean that set theory cannot serve as a foundation for Mathematics, but it does make the point that the whole foundation project is a bit more subtle than one might expect, and that the cirularities which launched this digression are not really pathologies, but a manifestation of the fact that life is complicated.
I have taken various liberties in selecting these quotations - you may want to consult the original text.
The second attempt is my own, more pessimistic, view. I think that "meaning" (if it exists at all) comes from a very lengthy bootstrapping process; thus any attempt at mathematical foundations is hopeless.