(Prompted by reflection on this old answer, and its suggestion of the “harmlessness” of the axiom of regularity.)

In ZFC, one may justify the axiom of foundation (AF, aka the axiom of regularity) as being “convenient, and harmless”, as follows. If $V = (V,\epsilon)$ is a model of ZFC–AF, then its well-founded part $V_{wf}$ models ZFC. Moreover, ZFC–AF suffices for the standard proof of the well-ordering principle, so $V$ believes that every set is isomorphic to some von Neumann ordinal, which will lie in $V_{wf}$. Hence the inclusion $V_{wf} \hookrightarrow V$ underlies an equivalence of their (internal, large) categories of sets; so any “purely structural” statement should hold in $V$ if and only if it holds in $V_{wf}$. “Purely structural” can be made precise in various reasonable ways (e.g. any statement expressible in the language of categories/elementary toposes/higher-order logic/type theory, interpreted into set theory), and includes essentially all of ordinary mathematics, excluding only a few explicitly material-set-theoretic statements such as AF itself.

Summing up: given any model of ZFC–AF, one can replace it by its well-founded part, which is now a model of ZFC, believing the same purely structural statements as the original model. So relative to ZFC–AF, AF has no purely structural consequences — i.e. it’s not quite conservative, but pretty close to it.

However, this argument relied essentially on the axiom of choice! **Relative to ZF–AF, is AF still harmless, or does it have some structural consequences?** Precisely: is there some “purely structural” statement $\varphi$ (in one of the senses above, or some similar sense), such that $ZF \vdash \varphi$, but $ZF-AF \not \vdash \varphi$?

I’m also interested in the same question with ZF weakened to IZF, CZF, and similar theories. Over these of course AF should be replaced by $\in$-induction (a classically equivalent and constructively better statement), as described in this older question (which considers closely related issues to the present question, but doesn’t as far as I can see directly imply an answer here).