Going the other direction, it turns out that classical ZF (using collection + separation) can be embedded in the theory where you drop extensionality---everything in classical set theory is still possible in a sense even when extensionality is not available. The idea is that when one lacks extensionality, one may recover it by defining an equivalence of sets, namely, that of having the same members, but then one wants really to define sets as equivalent when they have equivalent members, and so in in a transfinite refining process of the equivalence relation. The result is that in any model of ZF- without extensionality, one can define a corresponding model of ZF.

This is proved in *Harvey Friedman*, **The consistency of classical set theory relative to a set theory with intuitionistic logic**, *J. Symbolic Logic* **38** (1973), 315--319. The main point of the article is not this extensionality issue, but rather to embed classical ZF in an intuitionist logical system.

So although the axiom of extensionality seems fundamental to set theory—and one can find many instances in the literature describing extensionality as the most fundamental axiom—in fact one does not need it to develop set theory.

The theory used in that article is a theory of the kind you seek, a set theory lacking extensionality.

Meanwhile, as Asaf Karagila points out in the comments below, one cannot expect to recover classical ZF in the version of ZF without extensionality, if this is axiomatized with replacment rather than collection. This is because Dana Scott proved that this version of ZF-E is interpretable in Z, and so the consistency strength is strictly less than full ZF, if consistent.

My perspective on this result is that it shows that we should not axiomatize ZF-E using only replacement, since Scott's model violates the collection axiom in the same way that the Zermelo universe $V_{\omega+\omega}$ violates collection. A similar issue arises in other weakenings of ZFC, such as in the case of ZFC-powerset and ZFC-foundation, where one should typically use collection+separation rather than replacement.