Here I answer your additional question about finitely presented groups. The answer is then no.
Affirmation. Only finitely many finitely presentable groups may have the same given Cayley graph.
This works as follows. Let $X_1$ be the Cayley graph of some f.p. group. By finite presentability, there exists $n_0$ such that gluing $k$-gons each time you have a $k$-cycle in $X_1$ with $k\le n_0$, the resulting polygonal complex $X$ is simply connected. Now consider the locally compact group $G=\mathrm{Aut}(X_1)=\mathrm{Aut}(X)$. A finitely generated group with Cayley graph $X_1$ is the same as a group with a simply transitive action on $X_1$, hence letting $K$ be the stabilizer in $G$ of some vertex $x_0$, it is the same as a subgroup of $G$ whose intersection with each coset $gK$ is a singleton; let $\mathcal{W}$ be the set of subgroups of $G$ with this property. Using the action on $X$, a standard argument shows the following: let $S_G$ be the set of elements of $G$ mapping $x_0$ to a neighbor of $x_0$; then for every $\Gamma\in\mathcal{W}$, the group $\Gamma$ admits a presentation using $\Gamma\cap S_G$ as set of generators and relators of length $\le n_0$$\le\max(2,n_0)$ (2 is necessary because of the possible generators of order 2, unless these are represented by double edges). Since there are only finitely many such presentations, we are done.
Note that the proof even shows that there are finitely many marked groups $(\Gamma,S)$ with a given Cayley graph, in the finitely presentable case.