Yes. You can even have uncountably many. One recipe is as follows: consider a group $G$ with finite generating subset $S$ and an extension $1\to F\to G'\stackrel{\pi}\to G\to 1$, with $F$ finite. Endow $G'$ with the generating subset $S'=\pi^{-1}(S)$. Then the Cayley graph of $(G',S')$ only depends on the Cayley graph of $(G,S)$ and of the cardinal of $F$: indeed it is obtained by replacing any vertex by a complete graph on $|F|$ vertices and replacing any edge by the corresponding bipartite graph.
Therefore what you need is to find $G$ and infinitely many non-isomorphic $G'$ with $F$ of fixed size. One way to do so is when $G$ admits a finitely generated central extension $\tilde{G}$ with center an infinite-dimensional vector space $Z$ over $\mathbf{Z}/p\mathbf{Z}$ for some prime $p$: then $Z$ admits continuum many hyperplanes $H$ and there are still (*) continuum many non-isomorphic groups among the $G'=\tilde{G}/H$.
(*) I use that if $G$ is a finitely generated group and $\mathcal{N}(G)$ is the set of its normal subgroups, and if we write $N\sim N'$ if $G/N$ and $G/N'$ are isomorphic groups, then the equivalence relation $\sim$ on $\mathcal{N}(G)$ has countable classes. (Standard exercise)