This is true. One reference for this is Theorem C.1.4 in Hindry-Silverman's "Diophantine Geometry: An Introduction" which says that for an abelian variety $A$, number field $K$ and any prime $\mathfrak p$ above $p$, the reduction map $A(K)\to A(O_K/\mathfrak p)$ is injective on prime-to-$p$-torsion. The result you ask for follows since $\overline{\mathbb Q}$ is a union of number fields. One way to prove the required injectivity follows from the description of the kernel of the reduction map over the completion $K_{\mathfrak p}$, which can be identified with the formal group associated to $A$. Formal groups over $p$-adic fields have no prime-to-$p$ torsion.