If your category $A$ is AB5, the answer is positive, because the ind-torsion objects are only the objects $X$ of $A$ such that the canonical map from the colimit $_\infty X$ of $_n X$ to $X$ is an isomorphism, where $_n X$ is the kernel of multiplication by $n$ on $X$ and the (filtered) colimit is taken over positive integers $n$ ordered by divisibility. (The only thing to check is stability under extensions of this class of objects; thanks to AB5 the proof is the same as for abelian groups.) If $M$ is such that $n.Id_M$ is a mono for each integer $n>0$ (it is true for any subobject of a uniquely divisible object), then $M\to\, _n X$ is always zero, so $M\to\,_\infty X$ is always zero because $A$ is AB5.
I guess that AB4 is not enough.