I think of the axiom of regularity along with the axiom of extensionality as formalizing what I mean by "set". Once upon a time, before paradoxes, one could think of sets as just any collection of things. Unfortunately, axioms based on that picture, in particular the unrestricted comprehension axiom, led to contradictions, so it became clear that the original, contradictory notion of "set" must be replaced by something clearer. (People might have thought the original notion was perfectly clear, but the paradoxes show that it isn't.) The clearer picture that emerged (in a development beginning with Russell's type theory, and continuing through simple type theory) is of a cumulative hierarchy, in which sets are obtained as follows.

Begin with some non-set entities called atoms ("some" could be "none" if you want a world consisting exclusively of sets), then form all sets of these, then all sets whose elements are atoms or sets of atoms, etc. This "etc." means to build more and more levels of sets, where a set at any level has elements only from earlier levels (and the atoms constitute the lowest level). This iterative construction can be continued transfinitely, through arbitrarily long well-ordered sequences of levels.

This so-called cumulative hierarchy is what I (and most set theorists) mean when we talk about sets. A set is anything that is formed at some level of this hierarchy. This meaning of "set" has replaced older meanings.

The axiom of regularity is clearly true with this understanding of what a set is. It expresses the idea that the stages of the cumulative hierarchy come in a well-ordered sequence. (Without well-ordering, the instructions for each level, namely "form all sets whose elements are at earlier levels," would not be an inductive definition but a circularity.)

Although there are set theories that contradict regularity, I would say that any such theory (and also any theory that contradicts extensionality) is not about sets but about some different (though presumably similar) entities.