(Remark: I've edited the original answer to match the change in terminology, and I added some new comments.)

This is a theorem of Rosenstein from the paper *[$\aleph_0$-categoricity of groups][1]*. But note that he uses the terminology *bounded order* rather than *finite exponent*.
 

I say that a group is *$\aleph_0$-categorical* if its complete theory in the language of groups is $\aleph_0$-categorical (i.e., has a unique countable model). 

It is not hard to show that an $\aleph_0$-categorical group has finite exponent. This is also Theorem 1 of Rosenstein's paper. Theorem 2 of his paper provides a converse for abelian groups. So we have:

>**Theorem.** (Rosenstein 1971)<br/>
>An infinite abelian group is $\aleph_0$-categorical if and only if it has finite exponents.

------

**Appendix.** The discussion below has inspired me to add some discussion of the proof of theorems, and some other remarks. The theorem above comes from two results in Rosenstein's paper. 

**Theorem 1.** *An $\aleph_0$-categorical group has finite exponents.*

*Proof.* Rosenstein uses the following fact, due to Ryll-Nardzewski/Enegler/Svenonius independently.

> A countably infinite structure $M$ is $\aleph_0$-categorical if and only if for all $m>0$, the action of $\operatorname{Aut}(M)$ on $M^m$ has finitely many orbits. 

Now suppose $\operatorname{Th}(G)$ is $\aleph_0$-categorical. We may assume $G$ is countable. Since elements of distinct orders are in distinct orbits of the action of $\operatorname{Aut}(G)$ on $G$, there is a uniform bound on the orders of torsion elements in $G$. Moreover, $G$ has no elements of infinite order since, if $g$ were such then $(g,g^n)$  for varying $n$ would be in distinct orbits in the action on $G^2$. 

**Theorem 2.** *An infinite abelian group of finite exponents is $\aleph_0$-categorical.* 

See Rosenstein's paper. The main tool is a structure theorem for an abelian group of finite exponents as direct sums of cyclic groups. This is apparently called Prufer's First Theorem.

Some remarks:

 1. By YCor's comment about $\mathbb{Q}^{(\omega)}$, one cannot prove Theorem 1 by considering only orbits of singletons. ($\mathbb{Q}$ works also.)  

 2. It is easier to prove that an $\aleph_0$-categorical *torsion* group $G$ has finite exponents. Indeed, if not then by compactness/DLS there is a countable model of $\operatorname{Th}(G)$ with an element of infinite order, which cannot be $G$. But if $G$ is not a torsion group, I don't see a quick way to avoid Ryll-Nardzewski or Omitting Types of some kind (although one can substitute other facts that use these results, e.g., in an $\aleph_0$-categorical structure, the algebraic closure of a finite set is finite). 

 3. An $\aleph_0$-categorical group need not be abelian. An example is the countably infinite extraspecial $p$-group (see Definition 5.15 [here][2]). On the other hand, there are many results in model theory along the lines of "$\aleph_0$-categorical plus some model-theoretic property" implies some abelian-like structure. For example, Bauer, Cherlin, and Macintyre showed that an $\aleph_0$-categorical superstable group is abelian-by-finite.

 4. An infinite group of finite exponent need not be $\aleph_0$-categorical. For example, take an infinite finitely generated group of finite exponents (like a Tarski monster). Indeed, by the more general result about algebraic closure stated above, if $G$ is $\aleph_0$-categorical then any finite subset of $G$ generates a finite subgroup.


  [1]: https://pdf.sciencedirectassets.com/272332/1-s2.0-S0021869300X03328/1-s2.0-0021869373900926/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEO7%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJHMEUCIHZ04EYzAqky510u1oBygW1At1j7x%2B5r3OV5TdolDRWPAiEA5ivtoZxXofL7aNGuQV2i3JLgT5h2rBKfjeb3thSqnpMqvQMIlv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARADGgwwNTkwMDM1NDY4NjUiDL%2BDlH2eCwH7WlCJGSqRA%2BswnBQPE5BKYbaRV2PyQ4aGGv5yanbZr%2BNsSeoh7bzkFnaCOBldZUi3QTAbvATcNeQ%2B5U3nhtK%2Bk1T9XAPK01KK%2BJz2CI4%2BbO9MexuSIFM51tIpIw4HGAlqKh0Vivde20HjMftCSsu2SnHVpAm%2F9Qmq4IA%2Bl0y9SCUpRAjEnHIv6KMXDK%2Ba8EtBTjKgEy%2FDr3BA3Yl9V6LgdQ%2B1nCLnIHTwU0OEzLH2NHg9U4%2FzAm1H%2Bijj0P1ZzH2iE%2FnhNrtLDFijudB0TBDdRvlHPuS%2BzXcG3GYwxpvXkAQOCHjSHrcXOSpq5U6H%2BIdGU5KRSHT3NQrWg8SrqmfSTHfgexwzaTzjIug7aCjPxcfwjM4pISJs5l0gIYxO7mvYRspQSAQ6AEUheMjA%2B7X4lA4LZkDCUqFiAeAj3XuwGBcFSW0gl%2FITLqj%2FlBLLm1pFYnCp5%2BJBbCwBtjNoECW%2Br1nT4siVY5Hw356%2ByI11jZ1ki3nddALe%2FJKftuQfZda%2F%2BbAww1vUFfgsXO3Act4WTwuq7w%2FyTEnLMInBuPgFOusBxl1bgaKdr6n6asqTO6i7L4enprFVlt50OxnckSPq8FXKPtuF%2FPCv6in%2BEFAruPqy4L6otfG7XRusxiYl6Nal6yWgomSVkqkj8CNf67%2BfIC4yA2voGnyWT1RbS5Ii8zerGCWNVKJFKQTdSUscpT%2B0VN06tAPKHux7QrfSx%2Fav%2BNiIfWJ%2Bx1hOEg7lNGgXOqwB2zZ0dHdddrgdpGtSJ%2Bj6qO%2F9vC%2BAbqFsNLUk9jfa5DQefkhswtRZqEuWn9E4C2qalYDkpJvSqFtOE0jfWA8ITQXZoLaGmAc%2F7v4FKEhKkFHOma0mYkEmfX92rQ%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200714T223940Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=ASIAQ3PHCVTYTHJVZKX3%2F20200714%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=a7ed2b321ded4639a7ae34495eb16ebd013b44f97aabdadd8add96cb93031b9b&hash=379d723ce72c0430fb0fccaa6082460b05aa68b6845798676ca4b968e3ec62aa&host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61&pii=0021869373900926&tid=spdf-987313b5-1422-4f5b-a9e4-054d8aa1e05a&sid=e191fc0874983346178b6c043a894d10dbc8gxrqa&type=client


  [2]: https://hal.archives-ouvertes.fr/hal-00657716v2/document