I can give you half of the answer, but the other half is wide open.  I will use the characterization of supersingular primes as those primes p for which the normalizer of Gamma<sub>0</sub>(p) in SL(2,R) acts on the complex upper half plane to yield a genus zero quotient.

The monstrous moonshine conjecture asserted the existence of an infinite dimensional graded representation of the monster satisfying some exceptional properties.  It was conjectured by Conway and Norton, and proved by Borcherds, using the representation constructed by I.  Frenkel, Lepowsky, and Meurman.  One can take the graded dimension of this representation to get a power series, and it is the q-expansion of the J-function.  Furthermore, the graded trace of any element of order n in the monster is the q-expansion of a genus zero modular function that is invariant under Gamma<sub>0</sub>(nh) for some h|(12,n).  One can conclude somewhat abstractly that the normalizer of Gamma<sub>0</sub>(p) in SL(2,R) has to be genus zero for any prime p dividing the order of the monster.

The proof I've seen that no other primes satisfy the genus zero condition does not seem to have anything to do with the monster.  Instead, it is a delicate construction by Mazur involving the Eisenstein ideal, combined with some computations by H. Lenstra.  I may be ignorant of more refined arguments developed in the last 30 years, though.