Why can't mathematics be formalised in terms of classes rather than sets? I've always been curious about the seeming compulsion to found mathematics upon sets, be it ZF(C) or some other system. Of course, there are other approaches these days like category theory and type theory (themselves inextricably linked by the Curry–Howard–Lambek correspondence), but these rather seek an entirely different approach. What I'm really getting at is: why can't we simply deal with (improper) classes? That is, throw away sets as reified mathematical objects and just deal with the classes implicitly defined by predicates in some logical system. Does this indeed make certain areas of mathematics inaccessible? Are there other problems I might not have considered? I would appreciate if someone could enlighten me here in a general way, albeit perhaps also with some specific cases within mathematical subfields.
 A: You ask 

Why can't mathematics be formalised in terms of classes rather than sets?

The answer is that it can, and several standard accounts do precisely this. In particular, Gödel himself did this, for his version of what is now known as Gödel-Bernays set theory GBC (or von Neumann-Gödel-Bernays set theory NBGC) has only classes, not sets, as fundamental objects, and all the axioms refer only to classes. (One then introduces the concept of set as a defined term, a special kind of class, namely, a class that is a member of another class.) So this seems to be a central case that develops the theory as you like, and many contemporary accounts of GBC, such as Mendelson's, also use Gödel's version with only classes.  
Meanwhile, it is also common, perhaps more common, to present the GBC theory as a two-sorted theory, as Bernays did, with both sets and classes. See page 14 of Kanamori's article Bernays and set theory for informed comparison and discussion. The two presentations of the theory are easily interpreted in one another, and so the difference is widely viewed as a mathematically unimportant cosmetic difference. 
A: Putting together your two remarks

just deal with the classes implicitly defined by predicates in some logical system.

and

we could envisage a universe of discourse being something like the naturals (recursively defined), with only first-order classes.

seems to me to yield an informal description of full second-order arithmetic, $Z_2$.
As to whether mathematics can be formalized in such a system, the answer is yes, virtually all mainstream mathematics can even be formalized in weak subsystems of $Z_2$. See the book Subsystems of Second Order Arithmetic by Stephen Simpson.
