There are many equivalent ways to define supersingularity for an elliptic curve over a characteristic $p$ field.  One of them is that the $p$-torsion of the curve is connected, i.e., it is a purely infinitesimal group scheme of order $p^2$.  As Jonah mentioned, supersingular means very special, and is not a statement about smoothness.  There is a theorem of Deuring that implies the j-invariant of a supersingular elliptic curve always lies in $\mathbb{F}_{p^2}$, and as a consequence, all such curves are defined over a finite degree extension of $\mathbb{F}_p$.  

There are two notions of supersingular prime: one is relative to a fixed elliptic curve over $\mathbb{Q}$, and one is absolute.  For any elliptic curve $E/\mathbb{Q}$, a prime $p$ is supersingular for $E$ if $E$ has good supersingular reduction at $p$.  Such primes are known to be asymptotically density zero, but infinite in number (by a theorem of Elkies).  Lang and Trotter conjectured that the number of supersingular primes less than $N$ limits to a constant times $\sqrt{N}/\log(N)$ as $N$ gets large.

Supersingular primes in the absolute sense are those primes $p$ for which all supersingular elliptic curves over an algebraic closure of $\mathbb{F}_p$ have $j$-invariant in $\mathbb{F}_p$ instead of just $\mathbb{F}_{p^2}$.  These happen to be the primes that divide the order of the monster simple group, and they are also the primes for which the normalizer of $\Gamma_0(p) \in SL_2(\mathbb{R})$ acts on the complex upper half plane with a genus zero quotient. For general $p$, this normalizer contains $\Gamma_0(p)$ as an index 2 subgroup, with the nontrivial coset called the "Fricke involution" (a special case of Atkin-Lehner involultion).  There is a standard order 2 representative, taking $z \mapsto -1/pz$.  The quotient curve classifies unordered pairs of elliptic curves with dual degree p isogenies between them.  <strike>I do not know any canonical relations between these characterizations of supersingularity.</strike>

<b>Edit:</b> Thanks to Emerton for pointing out the connection.  I'll try to expand on it a bit.  The moduli problem of generalized elliptic curves with $\Gamma_0(p)$-structure has a coarse moduli space that is a smooth irreducible curve away from p, but has mod p fiber given by taking a disjoint union of two copies of $X(1)$ (a genus zero curve) and gluing along supersingular points (this description is more or less in Katz-Mazur, chapter 13).  A geometric point describing an elliptic curve with j-invariant $\alpha \in \mathbb{F}_{p^2}$ is glued to a geometric point on the other irreducible component describing an elliptic curve with j-invariant $\alpha^p$.  The Fricke involution switches the components, so the quotient of $X_0(p)$ by this involution is a genus zero curve glued to itself at finitely many supersingular points.  The quotient has arithmetic genus zero if and only if all supersingular geometric points are glued to themselves - otherwise, the flat modular deformation to characteristic zero yields a smooth curve of higher genus.  In other words, it is necessary and sufficient that all supersingular geometric points have no Frobenius conjugates, i.e., that the j-invariants of all supersingular curves lie in $\mathbb{F}_p$.

<b>More Edit:</b> I should give a more honest reply to Mariano's question, which was originally raised by Ogg in the mid 1970s (and he famously offered a bottle of Jack Daniels to anyone who could solve it).  Half of the question has an answer.  If we combine the results of Borcherds's paper [Monstrous moonshine and monstrous Lie superalgebras][1] with the results of the paper <i>Modular equations and the genus zero property of moonshine functions</i> by Cummins and Gannon, we get the following fact:

> Let G be a finite group acting faithfully on a conformal vertex algebra V by conformal symmetries, and suppose V has central charge 24 and character $\operatorname{Tr}(q^{L_0-1}|V) = j(\tau)-744$.  Then for any element $g \in G$, the series $\operatorname{Tr}(gq^{L_0-1}|V)$ is the q-expansion of a modular function that is holomorphic on the upper half plane, invariant under a discrete group $\Gamma \subset PSL_2(\mathbb{R})$ satisfying $\Gamma \supset \Gamma_0(N)$ for some $N$, and generates the function field of the quotient curve $\Gamma \backslash \mathbf{H}$.  In particular, the quotient curve is genus zero.

The monster simple group arises in this context because I. Frenkel, Lepowsky, and Meurman constructed a conformal vertex algebra satisfying the above hypotheses, whose group of conformal automorphisms is the monster.  The fact given above implies that for each prime p dividing the order of the monster, the quotient $X_0^+(p) = X_0(p)/\langle w_p \rangle$ is genus zero, where $w_p$ is the Fricke involution.  In particular, each prime dividing the order of the monster is necessarily supersingular.

I know some people who would like there to be a conceptual (read: non-enumerative) explanation for why <b>all</b> supersingular primes divide the order of the monster.  So far, the best I've heard is that the monster is really big, while there aren't that many supersingular primes.

  [1]: http://math.berkeley.edu/~reb/papers/index.html