If your category $A$ is AB5, the answer is positive, because the ind-torsion objects are only the objects $X$ of $A$ such that the canonical map from the colimit $_\infty X$ of $_n X$ to $X$ is an isomorphism, where $_n X$ is the kernel of multiplication by $n$ on $X$ and the (filtered) colimit is taken over positive integers $n$ ordered by divisibility. (The only thing to check is stability under extensions of this class of objects; thanks to AB5 the proof is the same as for abelian groups.) If $M$ is such that $n.Id_M$ is a mono for each integer $n>0$ (it is true for any subobject of a uniquely divisible object), then $M\to\, _n X$ is always zero, so $M\to\,_\infty X$ is always zero because $A$ is AB5.

AB4 is not enough: for any prime $p$ the additive group of $p$-adic integers, which belongs to the subcategory of abelian groups which is generated by torsion ones and is stable under products, imbeds into the uniquely-divisible abelian group of $p$-adic rationals. So, dualise and look at the opposite category of abelian groups.