As a set theorist, I feel some obligation to offer an answer here. First, the difficulties students may have in proving set theoretic containments like the one you mention above or in constructing $\epsilon$-$\delta$ proofs is not a matter of them struggling with set theory but rather of them struggling with something new: constructing a proof. In the case of $\epsilon$-$\delta$ proofs, a large part of this difficulty is in understanding quantifiers and how they work (for instance "for all ... exists ..." is not the same as "exists ... for all ...". This is because math is not a trivial subject to learn and some difficulty is required as the students minds stretch and grow. Surely there is no way around this.
Set theory provides an exact standard by which to discuss questions like "is there a subset of the real line which is uncountable but not of cardinality $|\mathbb{R}|$" (Hilbert's First Problem) or, maybe better, "is there an almost free, non free group?" (Whitehead's problem) or "If $h$ is a homomorphism a commutative Banach algebra into $C[0,1]$, is $h$ continuous?" (the negation being Kaplanski's conjecture). With the exception of the first question, these were asked, to my knowledge at least, without any thought that there was a foundational issue involved. Surely these are questions which could reasonably be asked regardless of how one sets up their foundations. To my knowledge category theory has never resolved these questions; set theory has in as satisfactory a manner possible (or at least until we adopt a more complete set of axioms). Now, one can argue at length about whether such questions are asked in poor taste or whether we should allow them to be asked at all. Readers interested in the question of "why care about set theory" should take a look at this (which might have been titled "why care about the uncountable").