As Andreas Blass pointed out, the meta-theory can be ordinary mathematics, at least in theory. In practice, without an explicit meta-theory, authority figures decide what is allowed, and what not. Tarski (like Cantor before him) learned this lesson the hard way, as can be read in accounts of Tarski's theorem about choice from 1924:
... when he tried to publish the theorem in Comptes Rendus de l'Académie des Sciences Paris, Fréchet and Lebesgue refused to present it. Fréchet wrote that an implication between two well known propositions is not a new result. Lebesgue wrote that an implication between two false propositions is of no interest.
It is no surprise that the modern notion of model and meta-theory are due to Tarski (and his colleague Robert Vaught) from 1956. But Tarksi already presented a "non-modern" notion of meta-theory in 1933, see the SEP entry on Tarski's Truth Definitions.
The issue is a little bit complex, but here is my own summary of the difference between those two notions for a start:
If I understood it correctly, for the 1933 version, the model (i.e. the algebraic structure about which we talk) is part of the meta-language and not mentioned separately. The assignment of objects to variables on the other hand is what can satisfy a given formula. A formula is (defined to be) true if it is satisfied by all possible assignments of objects to variables.
The 1956 version is treated less explicitly in the linked SEP entry, but it is hinted at that the model is no longer an implicit part of the meta-language, but an explicit object from set-theory. A model can satisfy a given formula (or sentence), similar to how an "assignment of objects to variables" could satisfy a given formula for the 1933 version. But the text also hints that the 1956 now relies stronger on an underlying set-theory, while the 1933 explicitly tried to minimize "the set-theoretic requirements of the truth definition".
