Anytime I see an $n!$ in some formula, my instinct is to look for the symmetric group on $n$ letters coming in somewhere. I have never done this seriously with the $n!$ in Taylor's theorem.

Question: Is there some way to see the $n!$ in Taylor's theorem coming naturally from a symmetry group?

Possible lead:

Here is a definition of $f^{(n)}(a)$ which does not depend on finding earlier derivatives: Let $g: \mathbb{R}^n \rightarrow \mathbb{R}$ be defined by $g(x_1,x_2,x_3, ..., x_n)$ is the lead coefficient of the unique $n^{th}$ degree polynomial passing through $(a, f(a)), (x_1, f(x_1)),(x_2, f(x_2)),...,(x_n, f(x_n))$. Then $f^{(n)}(a)$ is $1/n!$ times the limit of $g(x_1,x_2,x_3, ..., x_n)$ as $(x_1,x_2,x_3, ..., x_n)$ approaches $(a,a,a,...,a)$. Could the $1/n!$ be related to the symmetry of $g$ under exchange of coordinates?