This is indeed well-known to be not true, but almost true. If $E$ has no complex multiplication (that is to say if $\operatorname{End}_F(E)=\mathbb Z$), a good deal more is actually true.

>Theorem (Serre, 1972): Assume $E/F$ is without complex multiplication. Then the Galois representation $\rho_p:\operatorname{Gal}(\bar{F}/F)\longrightarrow\operatorname{Aut}(E[p])$ is surjective for all $p$ except a finite number.

The fact that all $p$ except possibly a finite number are such that $\rho_p$ is irreducible, not to mention the fact that there are infinitely many $p$ such that $\rho_p$ is irreducible, is significantly easier as it follows quite rapidly from the theorem of Shafarevich on the finiteness of the set of isomorphism classes of $F$-isogenous elliptic curves (see *Abelian $\ell$-adic Representations and Elliptic Curves* Jean-Pierre Serre, 1968).

However, if $E$ has complex multiplication by a quadratic imaginary subfield $K$ of $F$, consider $p$ a prime splitting as $\pi\bar{\pi}$ in $K$. Then $E[\pi]$ is a set of $F$-rational points of order $p$. In that case, there is a thus an infinite number of prime such that $\rho_p$ is reducible.

On that topic, the articles of Agnès David and Nicolas Billerey are recommended.