Historically, mathematics did not begin with set theory, and most working mathematicians today do not really care about foundations. Furthermore, it is known that not much axiomatic strength is actually needed for undergraduate math:

The body of research in reverse mathematics has established that weak subsystems of second-order arithmetic suffice to formalize almost all undergraduate-level mathematics. http://en.wikipedia.org/wiki/Reverse_mathematics

Some math is eternal. Low-level number theory, algebra, geometry, and analysis all have historical roots going much deeper than set theory or category theory or even formal logic itself. Let's not kid ourselves: As much as we may be devoted to them, Set Theory and Category Theory are ideologies that may not withstand the test of time. Ordinary mathematics can be formalized within them, but this is not the only way to go. The major players in the early 20th century French school of mathematics (Lebesgue, Borel, Baire, Poincare) all had severe reservations about set theory up to the ends of their lives. We should not deprive our students of essence of the beautiful eternal mathematics by forcing them to view everything through the lens of a modern ideology. Foundational studies can come later, after people have a good grasp on the unquestionable reality of basic mathematics.