Determining existence of a $p$-isogeny from $p|E(\mathbb{F}_{\ell})$

In Siksek's notes The modular approach to Diophantine equations he uses the following result:

Let $$p$$ be an odd prime. For an elliptic curve $$E$$ over $$\mathbb{Q}$$ if $$p|E(\mathbb{F}_{\ell})$$ then for all but finitely many primes $$\ell$$ then from Chebotarev's Density Theorem we conclude that $$E/\mathbb{Q}$$ has a $$p$$ isogeny.

Since he uses Chebotarev's Density Theorem the proof certainly involves the Galois representation $$\rho_{E,\ell}$$ but I struggle to see how

• Rather use $\rho_{E,p}$. Then the Frobenius at the varying $\ell$ has something to do with the question if $p\mid E(\mathbb{F}_{\ell})$. Aim at showing that the image is in a Borel subgroup. Commented Jun 18, 2021 at 16:40
• @JoeSilverman I mean a $p$-isogeny over $\mathbb{Q}$ Commented Jun 18, 2021 at 16:47
• @ChrisWuthrich How would it help that the image is in a Borel subgroup? Commented Jun 18, 2021 at 17:23
• The image is in a Borel subgroup if and only if there is a subspace of $E[p]$ fixed by Galois, which is the kernel of an isogeny defined over that field. Your local condition says that the Frobenius at $\ell$ maps to an element of $\operatorname{GL}(E[p])$ with $1$ as an eigenvalue. - I think this should help you to complete this. Sorry for not spelling it all out but I believe it helps you more if you complete it. Commented Jun 18, 2021 at 18:46