Polynomials are, essentially by definition, precisely the operations one can write down starting from addition and multiplication. More formally, polynomials with coefficients in a commutative ring $R$ are precisely the morphisms in the Lawvere theory of commutative $R$-algebras. So in some sense caring about polynomials is equivalent to caring about commutative rings and, more generally, commutative algebras. See, for example, this blog post for some details; in particular, that blog post makes precise the assertion that polynomials are not only the most natural but in fact the only natural operations on commutative $R$-algebras.