Why do categorical foundationalists want to escape set theory? This is a question that I have seen asked passively in comments relating to the separation of category theory from set theory, but I haven't seen it addressed in full.
I know that it's possible to formulate category theory within set theory while still being albe to construct the useful things one would want from category theory. So as far as I understand, all normal mathematics that involves category theory can be done as long as a little caution is taken. 
I also know that some people (categorical foundationalsists) would still like to formulate category theory without use of or reference to set theory. While I admit that I am curious about this for curiosity's sake, I'm not sure if there are any practical motivations for doing this. The only reason for wanting to separate category theory from set theory that I have read about is for the sake of `autonomy of category theory'.
So my question is twofold: What other reasons might categorical foundationalists have for separating category theory from set theory, and what practical purposes might it serve to do this?
 A: My opinion may not be of interest to readers here since I am a physicist by training and as such am most interested in how mathematics and reality come together.  That said, it is possible that my journey through foundations of mathematics, entwined with an interest in quantum gravity and especially background independence, perhaps sheds light in both directions.  After all, physics has been known to introduce interesting ideas to mathematics.
I think the short answer to the question of categories, sets and foundations comes from, as Todd pointed out, intrinsic meaning versus relational meaning.  For instance, I understand a functor from A to B to be the giving of meaning to some collection of objects of type A in terms of objects and morphisms of type B.  Contrast this with some of the axioms of set theory: 


*

*Two sets are equal (are the same set) if they have the same elements


We rely on some basic "knowledge" of the meanings of terms.  This meaning is provided by the reader.  Thus, we agree on the meaning of terms.
This relates to physics in the following way.  If I were God, I could see the universe from the outside.  Just as if I were reading that axiom 1, I would give meaning to all aspects of the universe.  Contrast this to the far more realistic vantage point of human beings who are decidedly embedded within the universe.  In this case, meaning comes not from an external viewer (the God looking at the set of all sets), but rather in how a change in one system induces a change in another system.  Think, for example, of a distant star, collapsing millions of years ago and turning slightly reddish.  This event is interpreted on earth on two photographic plates, or by a bit string being flipped from "white" to "red".
Rather than using Sets, which come with a sort of pre-arranged meaning, physics seeks to use a causal relation.  Further, this causal relation is seen as morphism in a category.  Up we go in terms of structure from nothing but "morphism structure" to Category-A structure.  The "meaning" is provided at each step by how the previous phase is interpreted in the latest phase.
More pointedly, in physics we cannot afford to build into the foundations of physics the ontology of set theory simply because it seems like a foundation for doing the math we love.
A: I don't agree that this is what (most) categorists who are interested in foundations are doing. 
It is true that Lawvere in the mid-60's (and perhaps to this day) wanted to develop a theory of categories independent of a theory of sets, but I don't think that represents the main thrust of modern-day categorical work on "foundations". Much more work has been directed toward developing a full-fledged categorical theory of sets, either as in Lawvere's Elementary Theory of a Category of Sets and extensions thereof, or understanding classical theories of sets such as ZF through a categorical lens, as in Algebraic Set Theory. There is also ongoing discussion of what strength of set theory is suitable for doing what category theorists would like to do. As one can see with even a casual perusal of such work, there is no antagonism toward set theory per se, or a desire to somehow get away from sets. 
I think some confusion might stem from over-hasty identification of set theory with a "canonized" form of set theory, such as ZFC (or something in that family such as Gödel-Bernays set theory), based on a single binary predicate called "membership". In ordinary ZFC, a set is characterized by its membership tree, so that the elements of sets are sets themselves, possessing their own internal structure. This may be termed a "materialist" form of set theory (material because elements of sets are considered as having "substance"). If there is antagonism toward this type of set theory on the part of some category theorists, it's because it lends itself to a conception of "set" that is largely irrelevant to the actual practice of core mathematics, insofar as mathematicians don't care what elements are "made of". 
The prevailing trends of mathematical practice today and throughout most of the twentieth century promote a more "structuralist" view: that what counts is not what the elements of a structure "are" particularly, but rather how they are interrelated in a structure, and where two structures are considered abstractly the same if they are isomorphic. This seems like a truism today, but it is precisely this view which drives a more categorically-minded view, which looks toward not what sets "are", but of how we use them, what abstract constructions we want to perform on them, and so on. Thus, concepts such as "power set" are in this view more relevantly captured by suitable universal properties which serve to characterize their structure up to specified isomorphism. A theory of sets which takes this point of view seriously and axiomatically may be termed a "structural set theory". 
Thus the real contrast is between "material" and "structural" theories of sets, with category theorists tending to prefer structural set theory. An example of such is Lawvere's aforementioned Elementary Theory of the Category of Sets (ETCS). A different and more recent example is Mike Shulman's SEAR (Sets, Elements, and Relations), which you can read about at the nLab. 
As for practical benefits of structuralist set theory: they are huge! It should be borne in mind that elementary topos theory was largely inspired by Lawvere's insight that Grothendieck toposes themselves model most of the axioms of the kind of structuralist set theory he was investigating in ETCS, and this has been revolutionary. This answer is already long enough, so I won't enter on a discussion of that here. 
A: Todd Trimble is right: not all categoriests who are interested in foundations reject set theory. It seems to me that category and set theory are deeply interrelated. Categorical problems, as the existence of some adjoint functors depends on some completions of ZFC. For example the statement every subcategory closed under limits of a locally presentable category is reflective (that is, the inclusion functor has a left adjoint) is equivalent to a set theoretical statement called Vopenka principle (see J. Adamek, J. Rosicky, Locally presentable and accessible categories, London Math. Soc. Lect. Notes, Cambridge University Press, 1994). There are lots of such problems, which depend on extensions of ZFC, both in category theory but also in other mathematical areas which are formulated categorical e.g. the Whitehead problem in abelian group theory: Is it true that every abelian group which has no nontrivial extensions through $\mathbb Z$ is projective (free)?   
A: One reason might be that some constructions and theorems in category theory need stronger axioms than just ZFC (the usual axioms for set theory).  For example, one might want to consider "functor categories," which don't exist in ZFC unless the domain category is small. 
Also, the usual characterization of "equivalence of categories" relies on class choice, which is stronger than just AC.
I haven't heard the term "categorical foundationalist" before, though, so this is just speculation. 
A: I honestly think that "naive category theory" (analogous to "naive set theory") helps me understand/remember/do mathematics far more than does set theory. And, since I do not intend to do "formal" category theory, I do not worry about its foundations.
I have some understanding of the Lawvere foundational program and related... but my own reaction to set theory as obligatory foundation (despite having great enthusiasm for foundations in my youth) is that it doesn't help anything.
At a different point in mathematical history, perhaps it was reasonable to worry that set theory was too volatile, or that mathematics more generally was so volatile that extreme set-theoretic scrupulousness was the only safe road, but by now many of the imagined dangers seem not realized.
And, if one does insist on foundations, I think category theory is a more dynamic, useful, helpful viewpoint than set theory. Category theory talks about what things do, rather than the atoms of which they're composed.
