I received my first glimpse of category theory (well, first glimpse after a friend's excited ravings about functors) from reading the first chapter of /Arrows, Structures, and Functors: the categorical imperative/ by Arbib and Manes. As I recall, it recasts basic set theory in the language of categories, and it's quite clear and readable with minimal prereqs. The rest of the book doesn't have much in the way of prereqs either but for some reason I didn't read much more of it until I was significantly older and more mathematically mature. This may in part be because the book takes a while to get around to doing functors. (Part of it also is that I soon realized that category theory was the study of arrow diagrams. My first exposure to abstract algebra, by self-study from Mike Artin's book, had induced in me a dislike of arrow diagrams -- some time later I realized that arrow diagrams are my friends, but Artin's book didn't encourage that sort of thinking.)
More generally, you may find introductions to category theory aimed at computer scientists (the book I mentioned above is one such) to be helpful.