As Will Sawin points out, the following is an answer to the question "Is it true that the set of primes such that $\rho_p$ is reducible is finite?" which is related but strictly stronger than the question actually asked.
That question admits a negative answer, but a positive answer if $E$ has no complex multiplication. If $E$ has no complex multiplication (that is to say if $\operatorname{End}_F(E)=\mathbb Z$), a good deal more is actually true.
Theorem (Serre, 1972): Assume $E/F$ is without complex multiplication. Then the Galois representation $\rho_p:\operatorname{Gal}(\bar{F}/F)\longrightarrow\operatorname{Aut}(E[p])$ is surjective for all $p$ except a finite number.
The fact that all $p$ except possibly a finite number are such that $\rho_p$ is irreducible, not to mention the fact that there are infinitely many $p$ such that $\rho_p$ is irreducible, is significantly easier as it follows quite rapidly from the theorem of Shafarevich on the finiteness of the set of isomorphism classes of $F$-isogenous elliptic curves (see Abelian $\ell$-adic Representations and Elliptic Curves Jean-Pierre Serre, 1968).
However, if $E$ has complex multiplication by a quadratic imaginary subfield $K$ of $F$, consider $p$ a prime splitting as $\pi\bar{\pi}$ in $K$. Then $E[\pi]$ is a set of $F$-rational points of order $p$. In that case, there is a thus an infinite number of prime such that $\rho_p$ is reducible.
On that topic, the articles of Agnès David and Nicolas Billerey are recommended.