I'll re-present the answer of Lumsdaine that both schemes are equivalent in Classical first order logic, but in a more direct manner.
I'll start with $\in$-induction to end up in Regularity scheme, using only
Classical First order logic with membership.
$\forall x [\forall y \in x (\varphi(y)) \to \varphi(x)] \to \forall x (\varphi(x))$, we negate both sides to get:
$\neg\forall x (\varphi(x)) \to \neg \forall x [\forall y \in x (\varphi(y)) \to \varphi(x)]$
$\exists x (\neg \varphi(x)) \to \exists x [\forall y \in x (\varphi(y)) \wedge \neg \varphi(x)]$
Let $ \neg\varphi \iff \psi $, then by rule of excluded middle: $\varphi \iff \neg \psi$, then:
$\exists x (\psi(x)) \to \exists x [\forall y \in x (\neg\psi(y)) \wedge \psi(x)]$
$\exists x (\psi(x)) \to \exists x [\psi(x)\wedge\not \exists y \in x (\psi(y)) ]$
Which is the regularity schema.
So both $\in$-induction and Regularity schema are reformulations of each other in classical first order logic with membership $\in$.
Also there is another way, albeit indirect, to show this equivalence.
Lets say that a predicate $\varphi$ is ascending if when it is fulfilled by all members of a set then it is fulfilled by that set itself (i.e. it meets the antecedent of $\in$-induction). On the other hand lets say that a predicate $\varphi$ is descending if when it is fulfilled by a set then there must be an element of that set that fulfills it. Now a property is said to be universal if it is fulfilled by all sets, and inhabited if it is fulfilled by at least one set.
- Regularity scheme say that all inhabited predicates are
non-descending.
- $\in$-induction scheme says that all ascending predicates are universal.
Now if we assume 1. then clearly any counter-example to 2. would be a descending predicate thus violating the consequent of 1.! So 1. implies 2.
Now if we assume 2. then suppose that $\varphi$ is inhabited and descending, then $\neg \varphi$ is ascending, and thus $\varphi$ would violate the consequent of $\in$-induction for predicate $\neg \varphi$. So 2. implies 1.
Thus $ 1. \iff 2. $
However this indirect argument of equivalence also requires the law of excluded middle, and so it is only valid in classical first order logic.