Inspired by the $\mathbf G_m$ case (Kummer theory), here is a positive result if you assume that $E(k)$ contains full $\ell$-torsion:
Lemma. Let $k$ be a field and $\ell$ a prime invertible in $k$, such that all finite separable extensions of $k$ have degree prime to $\ell$. Let $A$ be an abelian variety over $k$ such that $A(k)$ has full $\ell$-torsion, and let $P \in A(k)$. Then all points $Q \in A(k^{\text{sep}})$ with $[\ell] Q = P$ are defined over $k$.
Proof. The scheme-theoretic preimage of $P$ under $[\ell] \colon A \to A$ is naturally an étale $A[\ell]$-torsor over $k = \kappa(P)$, and the assumption on $A$ implies that the étale group scheme $A[\ell] \to \operatorname{Spec} k$ is constant. Then $H^1(k,A[\ell]) = \operatorname{Hom}^{\text{cts}}(\operatorname{Gal}(k^{\text{sep}}/k),A[\ell])$ is trivial by the assumption on $k$, so the torsor is trivial. $\square$.
For $\mathbf G_m$, the only alternative is that $k^\times$ has no $\ell$-torsion, in which case it is trivially $\ell$-divisible (as $[\ell]$ is invertible on $A(k)$). But on abelian varieties, there are intermediate cases with some but not all $\ell$-torsion, on which I have little to say at the moment.