Aside from any philosophical reasons, I think the pragmatic side of it is the reason they are so popular.

• They are easy to compute and deal with.

  a) before computers: differentiation, integration are easy. Also one of reason why Fourier series are popular.

  b) after computers: splines

• Most other functions (exponents, trig functions etc) are glorified polynomials, via Taylor series.

• Sometimes undeserved popularity: how many times have you seen a 15-term polynomial used in a regression equation?