As a set theorist, I feel some obligation to offer an answer here.  First, the difficulties students may have in proving set theoretic containments like the one you mention above or in constructing $\epsilon$-$\delta$ proofs is not a matter of them struggling with set theory but rather of them struggling with something new: constructing a proof.  In the case of $\epsilon$-$\delta$ proofs, a large part of this difficulty is in understanding quantifiers and how they work (for instance "for all ... exists ..." is not the same as "exists ... for all ...".  This is because math is not a trivial subject to learn and some difficulty is required as the students minds stretch and grow.
Surely there is no way around this.

That really has nothing to do with set theory so far.  In spite of a common misconception, set theorists do no actually care how it is that ordered pairs are defined.  Or how exactly one codes the notion of a function.  Set theory is not in competition with category theory, in spite of what category theory thinks.  A very good analogy is provided by computer science: set theory is machine language (or maybe better: a low level language like C) and category theory is object oriented programming.  While object oriented programming may provide a useful way of thinking about how to write a program, there still needs to be a machine language for the computer to run on.  Moreover, there are occasionally things for which it is just better (or even necessary) to code in a low level language.

Set theory provides an exact standard by which to discuss questions like "is there a subset of the real line which is uncountable but not of cardinality $|\mathbb{R}|$" (Hilbert's First Problem) or, maybe better,
"is there an almost free, non free group?" (Whitehead's problem) or "If $h$ is a homomorphism
a commutative Banach algebra into $C[0,1]$, is $h$ continuous?" (the negation being Kaplanski's conjecture).  With the exception of the first question, these were asked, to my knowledge at least, without any thought that there was a foundational issue involved.  Surely these are questions which could reasonably be asked regardless of how one sets up their foundations.
To my knowledge category theory has never resolved these questions; set theory has in as satisfactory a manner possible (or at least until we adopt a more complete set of axioms).  Now, one can argue at length about whether such questions are asked in poor taste or whether we should allow them to be asked at all.  Readers interested in the question of "why care about set theory" should take a look at <a href="http://www.math.cornell.edu/~justin/Ftp/utility_unctble.pdf">this</a> (which might have been titled "why care about the uncountable").