Let $G$ be a finitely-generated, torsion-free group. Under what circumstances is $Inn(G)$, the group of inner automorphisms $\phi(x)=g^{-1}xg$ of $G$, torsion-free?

Since $Inn(G)\cong G/Z(G)$, the group $G$ modulo its centre, this is certainly true if $G$ is abelian, or, at the other extreme, centreless. But when else?

Apologies if this is too elementary, I am a topologist who does not know the group theory literature as well as he would like.