'Category-theory'-free areas of pure math, 'category-theory'-loaded areas of applied math To put it short: In which active research areas of (pure) mathematics no (or only minimal) knowledge in category theory is required ?
To put it long: I know almost nothing about category theory - but I know that I do not like solving problems and providing arguments by diagram chasing and very "high-level" arguments.
 I'm currently getting near the end of my time as a student and want to start a PhD.    And while I'm not lacking a skill in algebra (which has been taught to us in an entirely category-theory-free manner), and therefor assume I could manage with some amount of category theory, I'm just not a fan of "high-level" arguments. So it's more a thing of personal taste. Therefore I would prefer to specialize in an area which does not exhibit too much category-theory (ideally: none).
Now the thing is, when reading around in this forum it seems to me that also in areas, in which I wouldn't expect category theory to come up, like analysis, it actually does come up ("Lipschitz categories").
Thus I would like to know, which sub(sub)fields of big fields like PDE, or number theory can be dealt with only a very small amount of category theory.     Especially: Are there any subfields of geometry that are category free ? I'm currently taking a course in differential geometry and to me it seems that category theory is (in disguise) almost everywhere, since our professor constantly explains how some diagram, that proves some assertion about manifolds is actually some category-theoretic notion.
When browsing for example through the work of Terence Tao suprisingly few diagrams and mentionings of "categories" come up, so it seems to me that certain areas of PDEs and number theory may fit the bill. But this perception may be due to that fact, that currently I understand almost nothing about what I read and therefore may have missed some categorical arguments.
Side question: Are there conversely any areas of applied mathematics that heavily use category theory ? Do there exist, for example, some applications of category theory on numerical mathematics ?
 A: Regarding your side question, more "applied" stuff that uses categories or category theory  is usually in the field of computer science. 
In fact a general category theory conference may have just as many people doing theoretic computer science as say people doing algebraic topology. 
The more applied theoretic computer science research that I've seen are people using categorical ideas in programming, databases, and quantum computation/information. I've even seen category theory in papers in cognitive science. However, I am not aware of any work using category theory in the numerical stuff.
A: Modern analysis is one large area where category theory has fairly minimal impact. Occasionally you see statements like "the category of locally compact Hausdorff spaces is dually equivalent to the category of abelian C*-algebras". I'm sure I could think of examples of more substantive uses of category theory, but they're pretty rare.
As a teacher, I've occasionally seen students with a background in category theory seemingly hindered by intuitions that don't work well (e.g., expecting the tensor product of Hilbert spaces to have the obvious universal property).
However, I agree with Todd's comment. If you ever find yourself in a situation where you need category theory for something you want to do, you are likely to suddenly find it much easier to pick up. At least, that's been my experience with other subjects that I thought I didn't like, until one day I needed them.
A: As a (slowly) recovering category-phobe, allow me to suggest that you change the way you think of category theory.  Specifically, don't think of category theory as a "theory".  A theory in mathematics generally consists of three components: a collection of related definitions, a collection of nontrivial theorems about the objects defined, and a collection of interesting examples to which the theorems apply.  To learn a theory is to understand the proofs of the main theorems and how to apply them to the examples.
Category theory is different: there is an incredibly rich supply of definitions of examples, but very few theorems compared to other established "theories" like group theory or algebraic topology.  Moreover, the proofs of the theorems are almost trivial (the Yoneda lemma is one of the most important theorems in category theory and it is not even called a "theorem").  A consequence of this is that you don't have to sit around reading a category theory book before you make contact with the language of categories: the very act of understanding how people express results from "ordinary" mathematics in the language of categories and functors is learning category theory.
So now I'll try to answer your question.  It is possible to work in nearly any area of pure mathematics without much category theory, and most areas have people everywhere on the category theory spectrum (with the possible exception of algebraic geometry, wherein the language of derived functors is basically built into the foundations).  Analysis in particular seems to be somewhat resistant (but not immune) to categorification, and if you are really committed to avoiding categories then you might consider exploring the more analytic aspects of what interests you (e.g. geometric PDE's or analytic number theory).  
But before you make that commitment, try to find some examples of categorical language in action in what you already understand.  The sheer ingenuity of it all might change your mind.
A: I would upvote Keerthi's comment multiple times if I could. Just find an area of mathematics that makes you smile and brings you happiness. If in the course of doing research you find that you need to deal with some category theory (say to study an important paper in the area), then you'll simply deal with it as it comes up, and at least you'll have some inherent motivation to do so. (And who knows? You might wind up finding some of it clean and attractive after all, and you'll have learned something to boot.) 
I can't resist adding something though, speaking as a category theorist. When I'm in the thick of "doing" some category theory, I'm almost never thinking "whoo, this is high-level stuff". Mainly, doing category theory just feels like doing a form of algebra; it's a very ordinary sort of activity. It can be very nice to realize, afterwards, how widely applicable this piece of algebra may be, but in the moment we're not thinking how mind-blowingly abstract it all is (and to me it just doesn't feel more abstract than other forms of algebra). 
We all have our tastes. I'll own up that I'm not a very geometric or analytic kind of guy -- I'm not greatly experienced in those modes of thought, but I'd hate for that to turn into the type of negative attitude so familiar from cocktail parties, where you meet someone who tells you (hearing that you do mathematics) that he always hated the subject, and seems almost proud of the fact. If on a given occasion I felt that I needed to know more geometry, then I hope I would simply learn what I needed to know, and not agonize over it. 
Edit: Andres Caicedo's comment under the question reminded me of something John Baez once wrote. (It's maybe a little snarky, but effective in a way Baez is renowned for.) 
