According to not necessarily reliable internet sources, Georg Cantor "told his colleagues and friends that he was proud of his choice of the letter aleph to symbolize the transfinite numbers, since aleph was the first letter of the Hebrew alphabet and he saw in the transfinite numbers a new beginning in mathematics: the beginning of the actual infinite."
Edit: According to less sketchy internet sources "The choice was particularly clever, as Cantor was pleased to admit, because the Hebrew aleph was also a symbol for the number one. Since the transfinite cardinal numbers were themselves infinite unities, the aleph could be taken to represent a new beginning for mathematics." from 'Georg Cantor and the battle for transfinite set theory' at http://ad.infinitum.simons-rock.edu/Dauben-Cantor.pdf, with footnote:
"Cantor explained his choice of the alephs to denote the transfinite cardinal numbers in a letter to Felix Klein of April 30, 1895. The original letter is in the Klein Nachlass, Universitatsbibliothek, Gottingen, and may also be read in a draft version in Cantor's letter-book for 1890-1895, pp. 142-143, also kept in the archives of the Niedersachsische Staats- und Universitatsbibliothek, Gottingen. See also Dauben 1979/1990, pp. 179-183; Meschkowski 1991, pp. 354-355."