This is an old question, but since no one seems to have given this answer, I thought I'd add it to the list, since it's where I first ran across Bernoulli numbers. Everyone (in mathematics) runs across the formulas $$ 1+2+\cdots+n=\frac{n(n+1)}{2}=\frac12n^2+\frac12n $$ and $$ 1^2+2^2+\cdots+n^2=\frac{n(n+1)(2n+1)}{6}=\frac13n^3+\frac12n^2+\frac16n. $$ One is naturally led to ask if $$ 1^k+2^k+\cdots+n^k $$ is also a polynomial in $n$. For a number theorist, there is no justification needed for looking at this question, but I'd suggest that this sort of sums of powers appears ubiquitously in mathematics, since a fundamental tool in studying just about any sequence is to look at its $k$'th moments. So now we're faced with the question of determining polynomials $P_k(T)$ satisfying $$ 1^k+2^k+\cdots+n^k = P_k(n). $$ Many areas of mathematics teach us that when confronted with a sequence (in this case, of polynomials), we should amalgamate them into a power series and study them simultaneously. In this case, a little experimentation shows that an exponential power series works nicely, so we look at $$ F(X,T) = \sum_{k=0}^\infty \frac{P_k(T)}{k!}X^k. $$ Then $$ F(X,n) = \sum_{k=0}^\infty \frac{P_k(n)}{k!}X^k = \sum_{k=0}^\infty \sum_{j=1}^{n} \frac{(jX)^k}{k!} = \sum_{j=1}^{n} e^{jX} = \frac{1-e^{(n+1)X}}{1-e^X}-1 = \frac{1-e^{(n+1)X}}{X}\cdot\frac{X}{1-e^X}-1. $$ The Taylor expansion of $\frac{1-e^{(n+1)X}}{X}$ is easy, so the interesting quantities that appear in the formula for $P_k(T)$ are the coefficients of your power series $\frac{X}{1-e^X}$. More generally, one defines Bernoulli polynomials whose coefficients are Bernoulli numbers and so that $P_k(T)$ is more-or-less a Bernoulli polynomial. Anyway, the moral is that the power series expansion of $\frac{X}{1-e^X}$ appears naturally even in the very elementary problem of finding a formula for a finite sum of $k$'th powers.
Just noticed that there's a closely related question Why do Bernoulli numbers arise everywhere? in which several people note that Bernoulli numbers occur automatically when looking at sums of powers. But I'll leave this answer, since it provides more details.
