The answer to the question in the title is No. You can prove this from the work of Wanda Szmielew on the elementary properties of abelian groups. This answer works for any kind of nonzero, bi-additive, binary multiplication (associative or not, commutative or not, unital or not).
In particular, an abelian group $A$ is elementarily equivalent to our favorite group $\mathbb Z$ iff $A$ is torsion free and of $p$-rank 1 for every prime $p$. The $p$-rank of $A$ is defined to be the minimum of $\textrm{dim}_{\mathbb Z_p}(A/pA)$ or $\omega$. An example of a torsion free abelian group of $p$-rank 1 for every $p$ is the subgroup $S\leq \mathbb Q$ consisting of rationals with square free denominator.
So $\mathbb Z$ and $S$ are elementarily equivalent. $\mathbb Z$ is the additive subgroup of the unital ring $\mathbb Z$, while (as Tom Goodwillie has pointed out) $S$ is not the additive subgroup of any unital ring. In fact, it is impossible to equip $S$ with any nonzero bi-additive multiplication. For, if $s, t\in S$, then $s$ and $t$ are $p$-divisible for almost all primes $p$. By bi-additivity, $s*t$ is $p^2$-divisible for almost all primes $p$. But the only element of $S$ that is $p^2$-divisible for almost all $p$ is 0, so $s*t=0$.