(Remark: I've edited original answer to match the change in terminology, and I added some new comments.)
This is a theorem of Rosenstein from the paper $\aleph_0$-categoricity of groups. But note that he uses the terminology bounded order rather than finite exponent.
I say that a group is $\aleph_0$-categorical if its complete theory in the language of groups is $\aleph_0$-categorical (i.e., has a unique countable model).
It is not hard to show that an $\aleph_0$-categorical group has finite exponent. This is also Theorem 1 of Rosenstein's paper. Theorem 2 of his paper provides a converse for abelian groups. So we have:
Theorem. (Rosenstein 1971)
An infinite abelian group is $\aleph_0$-categorical if and only if it has finite exponent.
Appendix. The discussion below has inspired me to add a sketch of the proof of the Theorem (from standard results), and some other remarks.
Theorem A. A countably infinite structure $M$ is $\aleph_0$-categorical if and only if for all $m>0$, the action of $\operatorname{Aut}(M)$ on $M^m$ has finitely many orbits.
Theorem B. Two countable abelian groups of finite exponent are isomorphic if and only if, for all $n>0$, they have the same number (possibly infinite) of elements of order $n$.
Theorem A is due to Ryll-Nardzewski/Enegler/Svenonius independently. I don't know the right attribution for Theorem B. One only needs to know that an abelian group of finite exponent is a direct sum of cyclic groups, which is apparently called Prufer's First Theorem.
Now the proofs (repeating my comment and YCor's comment below).
Theorem 1. An $\aleph_0$-categorical group is has finite exponent.
Proof. We use Theorem A. Suppose $\operatorname{Th}(G)$ is $\aleph_0$-categorical. We may assume $G$ is countable. Since elements of distinct orders are in distinct orbits of the action of $\operatorname{Aut}(G)$ on $G$, there is a uniform bound on the orders of torsion elements in $G$. Moreover, $G$ has no elements of infinite order since, if $g$ were such then $(g,g^n)$ for varying $n$ would be in distinct orbits in the action on $G^2$.
Theorem 2. An infinite abelian group of finite exponent is $\aleph_0$-categorical.
Proof. Suppose $G$ is an abelian group of finite exponent and $H$ is a countable group elementarily equivalent to $G$. Then $H$ is an abelian group of finite exponent having the same number of elements of every order as $G$ does, since all of this is in the first-order theory. So $G\cong H$ by Theorem B.
Some remarks on these proofs:
By another of YCor's comments (about $\mathbb{Q}^{(\omega)}$), one cannot prove Theorem 1 by considering only orbits of singletons. ($\mathbb{Q}$ works also.)
It is easier to prove that an $\aleph_0$-categorical torsion group $G$ has finite exponent. Indeed, if not then by compactness/LHS there is a countable model of $\operatorname{Th}(G)$ with an element of infinite order, which cannot be $G$. But if $G$ is not a torsion group, I don't see a quick way to avoid Ryll-Nardzewski or Omitting Types of some kind (although one can substitute other facts that use these results, e.g., in an $\aleph_0$-categorical structure, the algebraic closure of a finite set is finite).
The proof of Theorem 1 above is about the same as Rosenstein but he does say a little more. His proof of Theorem 2 takes up about a page and a half and it looks like this is because he is incorporating some of the content of the proof of Theorem B. (I did not read it very carefully though.)
Some final comments:
An $\aleph_0$-categorical group need not be abelian. An example is the countably infinite extraspecial $p$-group (see Definition 5.15 [here][2]). On the other hand, there are many results in model theory along the lines of "$\aleph_0$-categorical plus some model theoretic property" implies some abelian-like structure. For example, Bauer, Cherlin, and Macintyre showed that an $\aleph_0$-categorical superstable group is abelian-by-finite.
An infinite group of finite exponent need not be $\aleph_0$-categorical. For example take an infinite finitely generated group of finite exponent (like a Tarski monster). Indeed, by the more general result about algebraic closure stated above, if $G$ is $\aleph_0$-categorical then any finite subset of $G$ generates a finite subgroup.