Do hom-sets really live in the category Set?

Question

This isn't really a research-level question (sorry!), but I asked on math.se (link), and though the question was upvoted a few times, I didn't get any answers. So since there may well be more category theorists hanging out here, let me try again!

In familiar introductory books on category theory, one of the very first examples of a category given is Set. And what category is that?

Typically no explanation is given at this stage. But of course which category we are dealing with depends on our set theory. For an NF-iste, the category of NFsets has very different properties from the usual category Set (for a start NFsets is not cartesian closed). But fair enough, in an intro book you aren't going to mention that in Ch. 1! No: the authors are, surely, intending to point to stuff that their beginning readers can be assumed to know about, and are saying, "Hey you in fact already know about some categories, for example ..." The charitable reading, then, is that authors are relying on their readers to think of Set as comprising the sets they already know and love from their standard intro set-theory course.

Which are pure sets of the cumulative hierarchy -- pure in that there are no urlements, no memberless entities in the universe of sets other than the empty sets. [If set theories with urelements are mentioned in passing in an intro course, it is usually only to be dismissed and forgotten about.]

OK, then: in the absence of special explicit signals to the contrary, it seems (doesn't it?) that we might reasonably take the category Set mentioned in the very early pages to be a category of pure sets of the usual hierarchy. What else?

But then what are we to make, a bit later in the book, of e.g. the usual presentation of the Yoneda embedding as $\mathcal{Y}\colon \mathscr{C} \to [\mathscr{C}^{op}, \mathbf{Set}]$. Putting it this way assumes that hom-collections $\mathscr{C}(A, B)$ for $A, B \in \mathscr{C}$ actually live in $\mathbf{Set}$. And since such a hom-collection is a set of $\mathscr{C}$-arrows, that assumes that the $\mathscr{C}$-arrows must live in the world of pure sets too. [We may want the relevant hom-collections to be set-sized in the Yoneda embedding case -- but being no bigger than set-sized is one thing, living in the universe of pure sets is something else!]

But do we really want to assume that arrows [in the small-enough categories] are always pure sets? Isn't category theory supposed to be a story about how different bits of the mathematical universe hang together which doesn't presuppose some over-arching, all-in, set-theoretic reductionism, and so in particular doesn't presuppose from day one that all morphisms are pure sets??

Now, the foundational sections you often meet early in category theory books worry away about questions of size (sets vs classes etc.). But the present worry is orthogonal to all that, and is in a way more basic. If we want to make no assumption that the denizens of different bits of the mathematical universe are all cut from the same cloth, we won't want to slip into assuming that sets of these denizens are all pure sets. So in particular, do we really want to assume that a collection of arrows (hom-set) must live in $\mathbf{Set}$ -- where that's the category mentioned back almost on p.1 of the book -- (as opposed, perhaps, to being fully faithfully mappable into that world?

I guess there must be good discussions of this sort of thing in the literature somewhere, and I'm no doubt showing my ignorance by asking where! But, please, any pointers would be most gratefully received.

Answer 1

I think the question is vague and probably does not have a unique answer.

I would says that this kind of concern is actually not so different from the "size issue " generally presented by the set of objects, it's just that because this problem only arise under very weak set theoretical foundation this question is not mentioned in books.

In the same way that you don't care whether the "set of object" is actually a set or not, you don't really care whether the "sets of morphisms" are sets or not. what you need is "a notion of object" and a "a notion of morphism" that you can compose, but if you want to devise categories as algebraic structures then it's convenient to say that you have "a set of objects" and "a set of morphisms" but those "sets" of objects and morphisms does not really have to be elements of what you call the category of 'sets'. Although you would have to be careful at some point: certain theorem of category theory require to take limit and co-limit indexed by hom sets and might become false if you are not assuming that "hom-set" are actually sets. (for example the special adjoint theorem)

Now, any category theory book I've ever opened was working in ZFC and hence had no reason to care about this kind of questions. In fact, if we just want to drop the axiom of choice then a good thing to do would be to replace the notion of "functor" by the notion of "anafunctor" (which is a generalization of functor where for each object $X$, '$F(X)$' is well defined only up to canonical isomorphism) for which most theorems of category theory, like the fact that a fully faithfull and essentially surjective functor is an equivalence of category, remains true without the axiom of choice. And I don't know of any book of category theory which follows this point of view.