Set theory in practice This is more of a philosophy/foundation question.
I usually come across things like "the set of all men", or for example sets of symbols, i.e. sets of non-mathematical objects.
This confuses me, because as I understand it, the only kind of objects that exists in set theory are sets. It doesn't make sense to speak of other objects unless we have formalized them in terms of sets. So what to do with something like the set of all men? Are we working with a different set theory, a naive one? Or is it that we are omitting the formalization, because it is straightforward (e.g. assign a number to every man)?
 A: You don't have to define your objects as sets, in fact, you should avoid such unnatural definitions. I don't think a number theorist would be happy to see a proof referring to elements of a natural number or using the identity $1=\{0\}$. Such proofs are not acceptable because they won't survive even the slightest change in the foundations.
Similarly, if you develop Euclidean geometry, you don't define a point as a two-element set whose first element is a Dedekind cut and the second one is another weird set. You rather begin with axioms (either Euclidean ones or some axiomatization of the real line) and build the geometry on these. The set theory comes in if you want to show that your theory is consistent (as long as ZF is), and you do that by building a model within ZF.
In your example with men, your actually create a mathematical model of whatever you want to study, in the same way as physics does. There are always translation steps from real world to mathematics and back, they just happen to be trivial in this case. So it's not a problem that men are (modelled by) sets.
Only if you like to believe that mathematical objects do exist in some metaphysical sense, you will have a problem with the counter-intuitive claim that everything is a set. But you can just remove this axiom and stay agnostic about whether everything is a set or not. You will not lose anything - the only essential use of this axiom is that you are able to define the notion of equality rather than having it built into logic. And this is hardly of any importance outside the logic itself.
A: If you are being, say, at least semiformal in your approach to set theory, whether or not objects which are not sets exist depends upon the particular brand of set theory you choose.  The most common contemporary set theory, ZFC, is a "pure set theory", in which every object is itself a set, so the men indeed do not form a set.
But there are other set theories which allow non set elements, or urelements (what a great name!).  In particular, Quine's New Foundations with Urelements is a relatively popular such theory.
So far as I know it is towards the philosophical end of the spectrum to worry about whether sets should be allowed to contain urelements or not.  The mathematical justification for this is that, using the axiom of choice, any set can be put in bijection with a von Neumann ordinal, hence a pure set.  But you should be able to speak of sets of men if you want to, I suppose.  
Addendum: I like Sergei Ivanov's answer.  He hits the following key point: if you ask a generic mathematician whether or not an object which is not a set can be an element of a set, you will not get either "yes" or "no" as an answer, but rather an explanation of why they regard the question as being a mathematically unfruitful one.  When using sets for mathematical purposes, the "nature" of the objects which comprise sets is now regarded as being completely irrelevant.  This is the "structuralist" approach to mathematics, which has been clarified and taken further by the more modern categorical approach.  
