The relevant quantifiers and relations in mathematical axioms should be understood as predicate logic. In the case of ordinary first-order predicate logic, the membership operator $\in$ is defined as a binary relation over the universe, or domain of discourse, sometimes denoted $\Omega$.
Ordinarily, you can consider $\in$ to be a function and $\Omega$ to be a set of possible elements*. Yet as you've noticed, if you try to use mathematics founded in set theory (e.g. set-based domains and functions) to interpret set axioms, you'll introduce a form of circularity**.
One solution is to consider logic to be valid independent of mathematics. In the case of ZF or other systems, the axiomatization is first-order predicate logic. So long as you accept that first-order logic works, you don't need an interpretation*** in terms ofa mathematical functionsinterpretation.
Alternatively, you can consider sets as primitive and foundational to mathematics. ZFC is an example of how to interpret sets as primitive notions which are grounded asequivalent to objects in a formal logic and suitable as part of a foundation of mathematics. In this case, set axioms could be a description of non-foundational sets which are defined in terms of the primitive foundational sets used in definitions.
*Usually, objects in the domain of discourse could be anything in ordinary first-order logic, or for set membership, anything of which you could ask "is this a member of that set?". But in the context of mathematics it could be limited to defined mathematical objects, or in the context of pure set theory reduce to only sets.
**Circularity actually**Actually, circularity isn't necessarily a problem as long as the axioms are self-consistentsatisfiable.
***Unless you mean implementation to be something like computability