Marcel Berger, on p. 696 of his 2003 Springer book: A panoramic view of Riemannian Geometry, writes "Next we define the canonical musical duality", referring to a footnote: "These dualities are called musical because they are often written with symbols like $♭: V \rightarrow V^*$ and $♯: V^* \rightarrow V$."
Even if this is not the first metioning of "musical", we can assume that Berger invented that expression in one of his various writings, because in a panoramic viev of over 800 pages, a reference to another originator could have been expected.
Further Rob Kusner's hint here # or "sharp" is what Marcel Berger calls a "musical isomorphism" and Olivier Bégassat's comment here seem to provide sufficient evidence to determine inventor.
To answer this question completely, one would have to read through Berger's complete works. But if we wait for a while, this task will probably be reduced to touching a button.