The question was asked by a student, and I did not have a ready answer. I can think of the German word ``Einheit'', but since in German that is not how the identity element of a group is called, I doubt that is the origin. Any ideas?

13$\begingroup$ "but since in German that is not how the identity element of a group is called" ... Sometimes it is indeed called like this. Also the identity matrix is frequently or at least not rarely called 'Einheitsmatrix'. Another thought: Sometimes the identity element in a multiplicative group is called (perhaps sloppily) Einselement (where 'eins' means 'one'). $\endgroup$ – user9072 Feb 7 '12 at 14:09
Heinrich Weber uses Einheit and e in his Lehrbuch der Algebra (1896).

1$\begingroup$ That is almost certainly the origin, though it should be noted that one in Russian is "edinica". $\endgroup$ – Igor Rivin Feb 7 '12 at 14:58

4$\begingroup$ @Igor: The influential early textbooks on algebra tended to be written in German, unfair though that may be to those of us who grew up with English (or Russian). Quite a bit of common terminology and notation in mathematics seems to have originated in German work during the 19th century, such as the symbols
$K,k$
for fields. $\endgroup$ – Jim Humphreys Feb 7 '12 at 20:55 
2$\begingroup$ Well, Weber surely popularized the term. But his friend Dedekind used "einheit" before him to mean either a unit in a field, or a unit measure in geometry, and I'll bet if you look in his work you'll find it for groups. Probably if you dig into the 19th century you can find a series of earlier and earlier, vaguer and vaguer, uses of the term for a group identity. $\endgroup$ – Colin McLarty Dec 29 '12 at 17:35

4$\begingroup$ In todays German literature, it seems that Einselement (or even neutrales Element) is preferred over Einheit (which is used for units, i.e. invertible elements of a ring). $\endgroup$ – Hagen von Eitzen Feb 8 '14 at 21:49
The Encyklopädie article of Heinrich Burkhardt, Endliche discrete Gruppen (1899), p. 218 ascribes the origin thus:
15. Allgemeiner Gruppenbegriff. (...) die Gruppe enthält ein Element $e$, die Einheit $^{73)}$, das mit jedem andern $a$ $ae = a$ und $ea = a$ ergiebt; (...)
73) (...) G. Frobenius u. E. Stickelberger, J. f. Math. 86, 1879[78], p. 219.
The paper in question, Ueber Gruppen von vertauschbaren Elementen (1879), reads:
§.1. Definitionen.
Die Elemente unserer Untersuchung sind die $\varphi(\mathbf M)$ Klassen von (reellen) ganzen Zahlen, welche in Bezug auf einen Modul $\mathbf M$ incongruent und relativ prim zu demselben sind. (...) Das Element $\mathbf E$ (so bezeichnen wir im folgenden die Zahlenklasse, deren Repräsentant 1 ist) heisst das Hauptelement.

$\begingroup$ I'm confused. So, the conclusion is that the choice of E by Frobenius and Stickelberger is unmotivated (as their term for the unit is Hauptelement), and its identification with Einheit by later researchers (such as Burkhardt) is a false etymology? $\endgroup$ – Emil Jeřábek Apr 28 '15 at 11:04

$\begingroup$ @EmilJeřábek I agree that without further evidence of the transition, it's hard to tell how influential F & S's (pervasively used) E was in the eventual choice of e. They do tie it with eins insofar as their term for the unit is also Klasse von 1 (in quote above). $\endgroup$ – Francois Ziegler Apr 28 '15 at 23:23

$\begingroup$ For German speakers: am I missing something, or does Burkhardt's Encyklopädie really define a "group" as a cancellative semigroup, and then claim that the existence of the unit and inverses follow? $\endgroup$ – Tobias Fritz Jul 10 '15 at 17:30

$\begingroup$ @Tobias Fritz: No. First they define group as a group of permutations: "Hat eine Gesamtheit von Substitutionen die Eigenschaft, dass jedes Produkt von irgend zweien derselben selbst in ihr enthalten ist, so heisst sie eine Gruppe." (p. 211: Has a set of substitutions the property that every product of any two of them is again in this set, then this set is called a group). Then on p. 217, they define a general group by giving the axioms for a cancelative semigroup, and then state that if in addition the group is finite, inverses and an identity exist. $\endgroup$ – JanChristoph SchlagePuchta Jul 10 '15 at 18:13
The identity element for a complex number is (1,0)=1. The german word for "one" is "eins" so we write e.

3$\begingroup$ this seems speculative, compared with the sources quoted by Francois Ziegler $\endgroup$ – Yemon Choi Apr 26 '15 at 17:53