To my way of thinking, the most natural example of supervenience in mathematics---and the most similar to how this term is used in the philosophy of mind, where one uses it to describe the relation of the higher order properties, such as features of the mind, to the lower order properties, such as molecular structures in the brain---is provided by the sense in which set theory is viewed as forming a foundation for mathematics.
On that view of the foundations of mathematics (and there are many other views), the set-theoretic universe is seen to provide an ontological foundation for mathematics, in the sense that every mathematical object is regarded fundamentally as a set. One builds the natural numbers from sets as ordinals and then the integers and the rationals and the reals in any of the usual set-theoretic constructions; a group is a set with a binary operation (a set) having certain properties; a topological space is a set together with a set of subsets having a certain nature; and so on. On this view, every mathematical object is regarded as a set and the context of set theory is taken to provide a common forum in which to treat mathematical objects and constructions from what would otherwise be diverse forums. The existence of such a common forum allows us sensibly to apply knowledge from one area of mathematics to arguments in a distantly related area, and this is important.
So the view is that the basic features of the reals or of any mathematical object ultimately reduce to set theory in the sense that that object is fundamentally a set. But meanwhile, although this reduction of mathematics to set theory is important foundationally (and there are resulting a number of intriguing or even startling conclusions about ZFC-independence and paradox in non-set-theoretic contexts), the main view is also that the set-theoretic reduction is largely irrelevant for ordinary mathematics. We don't want to undertake most arguments in number theory or algebraic geometry or whatever with constant reference to the complete set-theoretic reduction of the subject, for example, by speaking of the "elements" of $\pi$. Thus, mathematics can be seen to reduce to set theory, but for most higher level mathematics, this reduction is either very complicated or not seen as illuminating of the interesting mathematical phenomenon at hand.
This relation seems very similar to the relation between our current understanding of mental properties and molecular structures in the brain. In principle, we believe that there is a reduction, but that reduction is either very complicated or not particularly illuminating of the mental phenomenon.
So this situation seems to accord accurately with your description of supervenience.
Addendum. Let me also mention another sense of supervenience, related to the point made by Gowers, in his second paragraph. The truth of a universal statement $\forall n\ \varphi(n)$ in arithmetic, say, reduces to the instances $\varphi(0), \varphi(1),\varphi(2)$, and so on. But by the Compactness theorem, one cannot prove the universal statement merely from those assertions in first order logic. Thus, the truth of $\forall n\ \varphi(n)$ would seem to supervene on those instances in the sense of the question. We don't prove a universal statement by proving each instance separately.