Let $(F,+,\cdot)$ be a rigid field, and put $M=(F,+,R)$, where $R(x,y,u,v)\iff xy=uv$. Then $(M,1)$ is interdefinable with $(F,+,\cdot)$ and therefore rigid, but $M$ has $|F|$ automorphisms $x\mapsto ax$ for each $a\in F^\times$. There exist rigid fields of any infinite cardinality, see e.g. http://mathoverflow.net/a/61082/12705https://mathoverflow.net/a/61082/12705.
Generalizing this and Eric Wofsey’s answer, if $(A,\cdot,R_1,R_2,\dots,F_1,F_2,\dots)$ is any rigid structure such that $\cdot$ is a group operation, we can take $M=(A,\cdot',R'_1,\dots,F'_1,\dots)$, where \begin{gather} \cdot'(x,y,z)=xz^{-1}y,\\ R'_i(x_1,\dots,x_n,z)\iff R_i(x_1z^{-1},\dots,x_nz^{-1}),\\ F'_i(x_1,\dots,x_n,z)=F_i(x_1z^{-1},\dots,x_nz^{-1})z. \end{gather} Then $M$ has automorphisms $x\mapsto xa$ for each $a\in A$, but $(M,1)$ is rigid. Elementary constructions of rigid structures (e.g., graphs) of arbitrary cardinality are easy to find, and expanding the structure with a $\cdot$ if necessary preserves rigidity.
Since all answers given so far are instances of the construction above, let me also mention an example that works in a different way. Let $F$ be a rigid field, and let $M$ be the polynomial ring $F[x]$. Since $F^\times=M^\times$ is definable in $M$, every automorphism of $M$ fixes $F$, and is uniquely determined by the image of $x$. Thus, $M$ has automorphisms $f(x)\mapsto f(ax+b)$ for $a\in F^\times$, $b\in F$, whereas $(M,x)$ is rigid.