This is indeed well-known to be not true, but almost true.
Theorem (Serre, 1972): Assume $E/F$ is without complex multiplication. Then the Galois representation $\rho_p:\operatorname{Gal}(\bar{F}/F)\longrightarrow\operatorname{Aut}(E[p])$ is surjective for all $p$ except a finite number.
The fact that all $p$ except possibly a finite number are such that $\rho_p$ is irreducible is actually much easier and follows quite easily from the theorem of Shafarevich on the finiteness of the set of isomorphism classes of $F$-isogenous elliptic curves.
However, if $E$ has complex multiplication by a quadratic imaginary subfield $K$ of $F$, consider $p$ a prime splitting as $\pi\bar{\pi}$ in $K$. Then $E[\pi]$ is a set of $F$-rational points of order $p$. In that case, there is a thus an infinite number of prime such that $\rho_p$ is reducible.
On that topic, the article of Agnès David and Nicolas Billerey are recommended.