The question was asked by a student, and I did not have a ready answer. I can think of the German word ``Einheit'', but since in German that is not how the identity element of a group is called, I doubt that is the origin. Any ideas?
The identity element for a complex number is (1,0)=1. The german word for "one" is "eins" so we write e.