Where did Zermelo first model the natural numbers by iterates of the singleton operator, and have the definitions been compared by himself? E. Zermelo is widely said to have modelled the (axioms of the) natural numbers by iterating the singleton operation $\{\cdot\}\colon \mathsf{Set}\rightarrow\mathsf{Set}$, $S\mapsto\{S\}$, whence the technical term Zermelo model.
J. von Neumann modelled the (axioms of the) natural numbers by iterating the successor operation ${\cdot}^+\colon \mathsf{Set}\rightarrow\mathsf{Set}$, $S\mapsto S\cup\{S\}$, 
whence the usual technical term von Neumann ordinal.

Incidentally, they both seem to have started at $S=\emptyset$, with von Neumann only writing `0' in his letter (the use of the Norvegian vowel started about a generation later).

Questions.
(0) Do you know a reference to original work of Zermelo's where the definition via iterates of $\{\cdot\}$ appears? 
(1) Are there scholarly references (or possibly, testimonies from people having attended those lectures) about whether Zermelo defined the natural numbers this way in his lectures in Göttingen and Zürich? 
(It of course seems unlikely that he did, having been much involved in axiomatic set theory, and having received what is now the standard definition from von Neumann.) EDIT: This question now has been copied to a more focused satellite question, which is where it should be answered. 
(2) Are there scholarly references about whether Zermelo ever publicly discussed, in lectures or in writing, von  Neumann's definition, possibly comparing his definition to his own? EDIT: This question now has been copied to a more focused satellite question, which is where it should be answered. 
(3) Are there mathematico-historical treatments in specialized journals of these two models? (This is not asking for a discussions, from a modern point of view, of properties of the two models in this thread, see remark below.) EDIT: If one is content with one example, and if a thesis counts as a "specialized journal", this question has been more or less answered by a reference kindly provided by Francois Ziegler in one of the comments. 
Remarks.


*

*Motivation for this question comes partly from research, partly from an expository writing project.

*I can read German, did have a look at some original publications of Zermelo's (though not for long...) prior to writing this question, and also did more than just scan for patterns like $\{\{\}\}$, but did not find any trace of the eponymous Zermelo model so far. (As I said, I did not read in his papers for very long.)

*Von Neumann's definition of the finite ordinals appears at least in a handwritten letter, dated 15. VIII. 1923, that von Neumann sent to Zermelo. This letter is reproduced in facsimile in a German mathematico-historical book by H. Meschowski, partially visible online. So the analogous question appears to be answerable about as nicely as one could possibly imagine. (Of course, it would be somewhat interesting learn that even in this case, Stigler's law of eponomy is validated, but it seems that it is not, and von Neumann ordinals really were first published by von Neumann.)

*Zermelo's definition appears to deemed technically inferior to von Neumann's in various well-known aspects (non-transitivity of Zermelo's sets, cardinality of Zermelo's sets being only 0 and 1, non-suitability of Zermelo's sets for defining limit-ordinals, etc). This is an interesting topic in and of itself, possibly fit for another MO-thread, but a bottomless pit and not the topic of this question, which is rather historical and reference-requestish; may the only reflection in this thread of theoretical considerations be through Zermelo himself, see question (3) above.

 A: Another go at the original publications turned up an easy, and probably widely-known, answer to question (0): Zermelo's model appears, even typographically identical to the usual form given today, at least on p. 267 of Math. Ann. Vol.65, No. 2 (1908). For completeness, here is the relevant paragraph:

Translation: 

Now if $Z'$ any other set of the kind required by the axiom, then there is a smallest subset $Z_0'$ of $Z'$ with the property under consideration, which corresponds to $Z'$ in exactly the same way as $Z_0$ corresponds to $Z$. But also the intersection $Z_0\cap Z_0'$, which is a common subset of $Z$ and $Z'$, must have the properties of $Z$ and $Z'$, and,  being a subset of $Z$, must contain the subset $Z_0$, and, being a subset of $Z'$, must contain the subset $Z_0'$. It thus follows by I that $Z_0\cap Z_0'=Z_0=Z_0'$, and that thus $Z_0$ is a subset of all possible sets which have the properties of $Z$, even though those sets need not be the elements of any set. The set $Z_0$ contains the elements $0$, $\{0\}$, $\{\{0\}\}$ and so forth. Let $Z_0$ be called the "sequence of numbers", because its elements can be used instead of number-symbols. This set is the simplest example of a "countably infinite" set (Nr. 36).

As further background, since it appears not too off-topic, here is part of von Neumann's 1923 letter:
(source is the book 
H. Meschkowski: Denkweisen [...] Vieweg, 1990, ISBN-13: 978-3-322-85074-4. DOI: 10.1007/978-3-322-85073-7
accessed with the help of a known search engine)

Transcript of the above part of the letter. (I am slighly unsure about the reading of some of the words, but will not record these instances of doubt, since there seems little point in doing so. The writing is sometimes a bit curious ("Grund Idee", "0Menge"). Also, von Neumann consistently (well, consistently-to-degree-two) uses five dots for what one might call the set of all finite ordinals except the first four finite ordinals, but uses fewer dots (namely four dots) for the larger set of what one might call the set of all finite ordinals except the three first finite ordinals .The difference in number of dots equals the difference in number of initial finite ordinals left out, somewhat suggesting that he did this intentionally. He is inconsistent, though, in that the number of horizontally aligned dots below $3$ is unequal to the number of dots behind $3$ within the curly bracket. 
Incidentally, this letter shows that "naive set theory" seems to have been a usual phrase far before Halmos' textbook.
Moreover, at least here, in this somewhat introductory writing, von Neumann defines limit-ordinals purely syntactically, not in the modern first-define-ordinal-as-transitive-and-well-ordered-by-the-membership-relation-then-divide-the-class-of-sets-so-defined-into-two-classes-the-successors-and-the-non-successors way. I do not know whether he used the latter more axiomatic definition in the full version of the work he is here introducing to Zermelo.
I also do not know what is the reference for the "naïve" treatment that von Neumann is announcing at the end of the quoted passage. 
So here is what seems an accurate transcription, to the point of doubly-dotting two of the i's:

In den beiden ersten Teilen [?] der Arbeit wird diese ganze Axiomatik auseinandergesetzt, und die Herleitung der bekannten Mengenlehre durchgeführt.
Das geschieht, schon der klaren Auseinandersetzung der angewandten Methode wegen, ziemlich weitgehend und detaillirt.
Da es sich größtenteils nur um die formalistische Herleitung bekannter Sätze handelt, mußte dabei viel triviales behandelt werden.
Neu sind (von einigen Kleinigkeiten abgesehen) in dieser Darstellung wohl nur die folgenden Punkte:
  
  
*
  
*Die Theorie der Ordnungszahlen (zweiter Teil, zweites Kapitel). 
  
  
  Es gelang mir die Ordnungszahlen auf Grund der Mengenlehre Axiome allein auf zu stellen.
  Die Grund Idee war die folgende:
  Jede Ordnungszahl ist die Menge aller vorhergehenden. So wird: (0 die 0Menge)
$0=0$,
$1=\{0\}$,
$2=\{0,\{0\}\}$,
$3=\{0,\{0\},\{0,\{0\}\}\}$,
.  .  .  .  .  .
$\omega= \{ 0,\{0\},\{0,\{0\}\},    \{0,\{0\},\{0,\{0\}\}\},.....   \}$,
$\omega+1= \{ 0,\{0\},\{0,\{0\}\},  .... ,   \{0,\{0\},....,\{0,\{0\}\}\},.....   \}\}$,
.  .  .  .  .
.  .  .  .  . 
(Für die positiven endlichen Zahlen lautet also die Regel so: $x+1 = x \overset{\cdot}{+}\{x\}.)$
Die Theorie hat auch im Rahmen der "naïven Mengenlehre" Sinn. (Sie wird, naïv  behandelt, demnachst in der Zeitschrift der Szegediner Universität erscheinen.)

For good measure, here is a translation of the above transcript. 

In the first two parst [word in the autograph is illegible but it is probably the German word "Teilen", an inflected form of "Teil"=part] of the work the axioms are explained in their entirety, and the deduction of the elements of the known set theory is carried out. 
This is done, if only to clearly explain the method applied, in a rather lengthy and detailed fashion.

["auseinandergesetzt" is, in this way of using it, (a participle of) a by now  obsolete German verb, meaning "to explain" or "to analyse"; similar remark for "Auseinandersetzung" which in contemporary German only means "row" or "conflict" but used to mean "analysis"]

Since for the most part this amounts to a formalistic derivation of known theorems, many trivialities had to be treated.
Only (disregarding a few small things) the following aspects are probably new:
  
  
*
  
*The theory of ordinal numbers (part two, chapter two). 
  
  
  I succeeded in founding the ordinal numbers on the axioms of set theory alone. The basic idea is the following: 
Each ordinal number is the set of all preceding ordinal numbers.
This implies (writing $0$ for the $0$set)

[see above for the formulas]

(For the positive finite numbers the rule therefore is:  $x+1=x\overset{\cdot}{+}\{x\}$.)
This theory also makes sense within the framework of "naive set theory". (It will, treated naively, soon be published in the journal of the university of Szeged.)

Remarks.


*

*It might help when trying to read Zermelo's paper to be told that Zermelo uses the somewhat unusual convention of closing a section introducing an axiom with that axiom's "name". I.e., he does not use "Axiom des Unendlichen" (i.e. "axiom of infinity") as a section-title under which he describes the axiom, but rather first describes the axiom and then, tombstone-like, closes the paragraph with "(Axiom des Unendlichen)".

*By "the axiom" in the above translation, Zermelo refers to the axiom of infinity. The "kind required by the axiom" is that this be a set with the two properties that (0), it contains 0 as an element, and (1), it is closed w.r.t. applying the singleton-operator.

*By "with the property under consideration" he to all appearances means two properties, namely, the two properties required by the axiom of infinity.

*The sets $Z$ and $Z_0$ in "as $Z_0$ corresponds to $Z$" he refers to two sets discussed one paragraph earlier (which I won't explain here, to keep a lid on things).

*The grammatical construction "in genau derselben Weise wie $Z_0$ dem $Z$ eine [...]" is one of the apparently rather rare examples where mathematical German is more concise than mathematical English, the latter lacking (for all I know) dative-inflected article (such as "dem").

*Translating "möge" as "Let" is the closest I can think of; it is (what I would call) an optative mood (though someone more versed in the theory of grammar may disagree-there seem to be subtypes of "optative").

*"Nr. 36" refers forward in the paper, to Section 36 on p. 280, where a proof is given that $Z_0$ is an "infinite" set in the sense that it has the property that it is isomorphic to a proper subset of itself, and that every set with this property must contain an isomorphic image of $Z_0$. (Zermelo appears not to mention Dedekind in the context of such considerations.)


Incidentally (I will not dwell on this since this is not the topic of this thread), the introduction of the above-cited paper contains an explicit mention of Zermelo's, to the effect that he could not yet rigorously prove the consistency of his axiom system, although he considers this question "sehr wesentlich" (and hints at deeper future work in that direction). 
So, somewhat interestingly, the consistency of ZFC has been called into question already in what arguably is its founding document.
Questions (1),(2),(3) of the OP, which are basically about the reception of von Neumann's definition, remain open(-ended).
