Consider first the Unrestricted Axiom of Comprehension

($\exists$y)($\forall$x)(x$\in$y $\leftrightarrow$ $\phi$(x))

and the resulting Russell paradox

y$\in$y $\leftrightarrow$ y$\notin$y

One can certainly understand the early set theorists' concern over the existence of a set y such that y$\in$y.


Consider also Cantor's proof found in his letter to Dedekind (found in van Heijenoort's "From Frege to Goedel" (pp113-117) that the system of "all numbers" (all ordinal numbers) $\Omega$ and its successor $\Omega^{'}$ are "inconsistent multiplicities".  In order for his 'proof' to work one must allow $\Omega$$\in$$\Omega$ and the infinite descending sequence ....$\in$$\Omega^{''}$$\in$$\Omega^{'}$$\in$$\Omega$ (obviously for the Burali-Forti paradox as well).


In Zermelo's paper "Investigations in the foundations of set theory I" (also found in van Heijenoort--pp. 199-215), one finds the Axiom of Foundation cropping up as the following "theorem":

"Every set $M$ possesses at least one subset $M_0$ that is not an element of $M$."

which he 'proves' using the Axiom of Separation.


A good introduction to the Axiom of Regularity and its philosophical  and historical underpinnings is the Wikipedia entry "Axiom of Regularity".


That having been said, a good (at least in my opinion) introduction to nonwellfounded set theory is the paper by Takashi Nitta, Tomoko Okada, and Athanassios Tzouvaras titled "Classification of non-well-founded sets and an application" which can be found under title on the Web.