Thinking about the distinction between language and metalanguage may be helpful here.  When one describes set theory as possessing a single binary relation denoted $\in$, one is operating at the level of metalanguage.  Specifying axioms satisfied by $\in$ is at the level of the language.  At this stage sets could be beer mugs as Hilbert famously said in a slightly different context.  

Next, one assumes the existence of a model of the language, and interprets the meaning of the language, or more precisely of the theory expressed in the language, in that model (no more beer mugs).  

In my experience, traditionally trained mathematicians (who have never taken a logic course) have great difficulty with the language/metalanguage and theory/model distinctions.  This is because some of them tend to think of mathematics as "one great monolithic thing" and introducing such dichotomies goes counter to that philosophy.  I don't think Paul Halmos ever overcame his suspicious attitude toward the standard dichotomies in logic; for details see [this 2016 publication in *Logica Universalis*](http://dx.doi.org/10.1007/s11787-016-0153-0).

As far as the OP's comment to the effect that "Philosophical analysis of the question is unhelpful" I would agree in the sense that there is a lot of unhelpful philosophy of mathematics out there; a sterling example is the work of Hide Ishiguro on Leibniz which manages to combine bad mathematics, bad history, and bad philosophy in a single chapter 5; see [this 2016 publication in *History of Philosophy of Science*](http://dx.doi.org/10.1086/685645).  On the other hand, the OP's problem with alleged "circularity" is based precisely on certain philosophical *partis pris* as I tried to suggest above.

Note 1. In response to the new version of the question that shifts the emphasis somewhat to functions and relations, note that it may be helpful to consult the article

>Leinster, Tom. Rethinking set theory. Amer. Math. Monthly 121 (2014), no. 5, 403–415

which seeks to present an accessible introduction to a category-theoretic approach to the foundations focusing on functions (instead of points and sets).