The answer is yes, provided ZF itself is consistent.
The reason is that the existence of the hyperreals, in a context with the transfer principle, implies that there is a nonprincipal ultrafilter on $\mathbb{N}$, because if $N$ is any nonstandard (infinite) natural number, then the set of all $X\subset\mathbb{N}$ with $N\in X^*$ is a nonprincipal ultrafilter on $\mathbb{N}$. Basically, the ultrafilter concentrates on sets that express all and only the properties held by the nonstandard number $N$. (See also my answer to A remark of Connes, where I make a similar point.)
Thus, in a model of ZF with no nonprincipal ultrafilter on $\mathbb{N}$, there is no structure of the hyperreals satisfying the transfer principle.