Update: more thoughts, including a shorter and more nonsensical "proof", on my blog.
Orde's paper, "
On Dirichlet's Class number formula", gives a beautiful nonsense proof, before giving a rigorous one. Thanks to KConrad for pointing out Orde's paper to me. Orde is a little terse, so let me expand:
Let $R$ be the ring of integers in $\mathbb{Q}[\sqrt{-p}]$. For simplicity, take $p>3$. For any positive integer $N$, let $S(N)$ be the number of ideals in $R$ with norm $N$. By unique factorization into prime ideals and a little thought, we have
$$S(N) = \sum_{d|N} \left( \frac{-p}{d} \right). \quad (*)$$
This formula is correct for $N>0$. Orde explains how to extend this formula to be correct for $N \neq 0$. The formula at $N=0$ will then be the class number formula!
Let $C$ be the class group of $R$. Let $Q$ be the set of integral quadratic forms with discriminant $-p$, modulo equivalence. Note that $Q$ is the disjoint union of $Q^{+}$ and $Q^{-}$; the positive definite forms and the negative definite ones. For most purposes, we discard $Q^{-}$, but today we want it around. There is a standard bijection between $C$ and $Q^{+}$.
For $c \in C$ and $N>0$, let $S_c(N)$ be the number of ideals of $R$ of class $c$ and norm $N$. So $S(N)=\sum_{c \in C} S_c(N)$. Let $q$ be the corresponding positive definite form and let $T_q(N)$ be the number of representations of $N$ be the form $q$. By the standard relationship between quadratic forms and ideals, $T_q(N)=2 S_c(N)$. (That $2$ is because $R$ has $2$ units.) Also, since $N>0$, we have $T_{-q}(N)=0$. So
$$\frac{1}{2} \sum_{q \in Q} T_q(N) = S(N) = \sum_{d|N} \left( \frac{-p}{d} \right) \quad (**).$$
The left and right hand sides of $(**)$ are symmetric in exchanging $N$ and $-N$, so $(**)$ is also valid for $N<0$.
Now, consider $(**)$ for $N=0$. For any $q \in Q$, we have $T_q(0)=1$, since $q$ is either positive or negative definite. So the left hand side is $(1/2) |Q|=|C|$.
Everything divides $0$, so the right hand side is $\sum_{d>0} \left( \frac{-p}{d} \right)$. That doesn't converge, but its Cesaro sum is $(1/p) \sum_{d=1}^{p} (p-d) \left( \frac{-p}{d} \right).$ (If we were doing the case that $p \equiv 1 \mod 4$, that average would be over $4p$ terms, instead of just $p$ of them.) So we "derive" that
$$|C| = (1/p) \sum_{d=1}^{p} (p-d) \left( \frac{-p}{d} \right).$$
This is easily shown to be equivalent to the class number formula.