There may be an earlier source, but Adolf Hurwitz 1897 is one upper bound:
Adolf Hurwitz, Über die Erzeugung der Invarianten durch Integration, Nachr. Ges. Wiss. Göttingen (1897), 71–90.
Hurwitz’s paper introduced and developed the notion of an invariant measure for the matrix groups SO(N) and U(N). He also specified a calculus from which the explicit form of these measures could be computed in terms of an appropriate parametrisation — Hurwitz chose to use Euler angles. This enabled him to define and compute invariant group integrals over SO(N) and U(N).
source: A. Hurwitz and the origins of random matrix theory in mathematics