An Elementary Theory of the Category of Sets
http://tac.mta.ca/tac/reprints/articles/11/tr11abs.html
I always had a problem with ZFC because it makes too many arbitrary choices: why do we choose this countable set to be the natural numbers and not this other one? Why do we choose Kuratowski ordered pairs instead of some other version? This paper turned me on to the idea that all of mathematics could be done in a "nice" way, where things are only determined up to unique isomorphism by the properties you want them to satisfy. It was also my first exposure to category theory, and so holds a special place in my heart.
