[Wiki, as already in the comments]

For the easy negatives, consider the Cayley graph with respect to the generating set $S=G$. The graph obtained is then a complete graph. It can be embedded in $\mathbb{R}^{G} = \{ (a_g)_{g \in G} \mid a_g \in \mathbb{R} \}$ as follows: $g \mapsto \delta_{g\cdot}$ where $\delta_{gh} = 1$ if $h=g$ and $0$ else. The edges between two vertices all have length $\sqrt{2}$.

The permutation matrices are linear isometris of $\mathbb{R}^G$ and will permute the vertices as desired. So the symmetries of the Cayley graph contain $S_{|G|}$ which is strictly larger than $G$ as soon as $|G| \geq 2$.

More generally, as Nick Gill pointed out, this embedding shows that (without bound on $m$, or in fact, as soon as $m \geq |G|$) there is no difference between the "abstract" automorphisms and the "geometric" automorphisms. Positive answers for "abstract" are found in this answer of C. Godsil (again, pointed out by N. Gill) http://math.stackexchange.com/questions/1098115/when-is-the-automorphism-group-of-the-cayley-graph-of-g-just-g

[Wiki, as already in the comments]

For the easy negatives, consider the Cayley graph with respect to the generating set $S=G$. The graph obtained is then a complete graph. It can be embedded in $\mathbb{R}^{G} = \{ (a_g)_{g \in G} \mid a_g \in \mathbb{R} \}$ as follows: $g \mapsto \delta_{g\cdot}$ where $\delta_{gh} = 1$ if $h=g$ and $0$ else. The edges between two vertices all have length $\sqrt{2}$.

The permutation matrices are linear isometris of $\mathbb{R}^G$ and will permute the vertices as desired. So the symmetries of the Cayley graph contain $S_{|G|}$ which is strictly larger than $G$ as soon as $|G| \geq 2$.

More generally, as Nick Gill pointed out, this embedding shows that (without bound on $m$, or in fact, as soon as $m \geq |G|$) there is no difference between the "abstract" automorphisms and the "geometric" automorphisms. Positive answers for "abstract" are found in this answer of C. Godsil (again, pointed out by N. Gill) https://math.stackexchange.com/questions/1098115/when-is-the-automorphism-group-of-the-cayley-graph-of-g-just-g

[Wiki, as already in the comments]

For the easy negatives, consider the Cayley graph with respect to the generating set $S=G$. The graph obtained is then a complete graph. It can be embedded in $\mathbb{R}^{G} = \{ (a_g)_{g \in G} \mid a_g \in \mathbb{R} \}$ as follows: $g \mapsto \delta_{g\cdot}$ where $\delta_{gh} = 1$ if $h=g$ and $0$ else. The edges between two vertices all have length $\sqrt{2}$.

The permutation matrices are linear isometris of $\mathbb{R}^G$ and will permute the vertices as desired. So the symmetries of the Cayley graph contain $S_{|G|}$ which is strictly larger than $G$ as soon as $|G| \geq 2$.

[Wiki, as already in the comments]

For the easy negatives, consider the Cayley graph with respect to the generating set $S=G$. The graph obtained is then a complete graph. It can be embedded in $\mathbb{R}^{G} = \{ (a_g)_{g \in G} \mid a_g \in \mathbb{R} \}$ as follows: $g \mapsto \delta_{g\cdot}$ where $\delta_{gh} = 1$ if $h=g$ and $0$ else. The edges between two vertices all have length $\sqrt{2}$.

The permutation matrices are linear isometris of $\mathbb{R}^G$ and will permute the vertices as desired. So the symmetries of the Cayley graph contain $S_{|G|}$ which is strictly larger than $G$ as soon as $|G| \geq 2$.

More generally, as Nick Gill pointed out, this embedding shows that (without bound on $m$, or in fact, as soon as $m \geq |G|$) there is no difference between the "abstract" automorphisms and the "geometric" automorphisms. Positive answers for "abstract" are found in this answer of C. Godsil (again, pointed out by N. Gill) http://math.stackexchange.com/questions/1098115/when-is-the-automorphism-group-of-the-cayley-graph-of-g-just-g