Let $f: X \to Y$ morphism between two quasiprojective, irreducible varieties over the complex numbers, such that the image of $f$ is Zariski dense in $Y$ and there is a Zariski dense subset $U$ (not necessarily open) of $X$ such that $f$ is injective on $U$.

Is $f$ then always a birational morphism? If not, is there a simple counterexample?