This is an old question, but since no one seems to have given this answer, I thought I'd add it to the list, since it's where I first ran across Bernoulli numbers. Everyone (in mathematics) runs across the formulas
$$ 1+2+\cdots+n=\frac{n(n+1)}{2}=\frac12n^2+\frac12n $$
and
$$
   1^2+2^2+\cdots+n^2=\frac{n(n+1)(2n+1)}{6}=\frac13n^3+\frac12n^2+\frac16n.
$$
One is naturally led to ask if 
$$ 1^k+2^k+\cdots+n^k $$
is also a polynomial in $n$. For a number theorist, there is no justification needed for looking at this question, but I'd suggest that this sort of sums of powers appears ubiquitously in mathematics, since a fundamental tool in studying just about any sequence is to look at its $k$'th moments. So now we're faced with the question of determining polynomials $P_k(T)$ satisfying
$$ 1^k+2^k+\cdots+n^k = P_k(n). $$
Many areas of mathematics teach us that when confronted with a sequence (in this case, of polynomials), we should amalgamate them into a power series and study them simultaneously. In this case, a little experimentation shows that an exponential power series works nicely, so we look at
$$ F(X,T) = \sum_{k=0}^\infty \frac{P_k(T)}{k!}X^k. $$
Then
$$ F(X,n) = \sum_{k=0}^\infty \frac{P_k(n)}{k!}X^k
    = \sum_{k=0}^\infty \sum_{j=1}^{n} \frac{(jX)^k}{k!}
    = \sum_{j=1}^{n} e^{jX}
    = \frac{1-e^{(n+1)X}}{1-e^X}-1
    = \frac{1-e^{(n+1)X}}{X}\cdot\frac{X}{1-e^X}-1.
$$
The Taylor expansion of $\frac{1-e^{(n+1)X}}{X}$ is easy, so the interesting quantities that appear in the formula for $P_k(T)$ are the coefficients of your power series $\frac{X}{1-e^X}$. More generally, one defines Bernoulli polynomials whose coefficients are Bernoulli numbers and so that $P_k(T)$ is more-or-less a Bernoulli polynomial. Anyway, the moral is that the power series expansion of $\frac{X}{1-e^X}$ appears naturally even in the very elementary problem of finding a formula for a finite sum of $k$'th powers.

---

Just noticed that there's a closely related question https://mathoverflow.net/questions/61252/why-do-bernoulli-numbers-arise-everywhere in which several people note that Bernoulli numbers occur automatically when looking at sums of powers. But I'll leave this answer, since it provides more details.