Let $p$ be a rational prime number, and let $K$ be a finite extension of $\mathbb Q_p$. Let $A$ be an abelian variety over $K$. For any rational prime number $\ell$, let $K(A[\ell])$ be the field of definition of the $\ell$-torsion points of $A$.

Suppose $A$ and $A'$ are $K$-isogenous abelian varieties over $K$.

Is there an odd prime number $\ell\neq p$ such that $K(A[\ell])=K(A'[\ell])$?

Are there even infinitely many such $\ell$?