Let $f:X \rightarrow Y$ be a flat surjective morphism of complex projective varieties with connected fibers, $X$ normal, $Y$ smooth and $dim (Y) < dim(X)$. Suppose $L \in Pic (X)$ is nef such that $L|_{X_{y}} \cong \mathcal{O}_{X_{y}}$ for all $y \in Y$. Then does there exist $L_Y \in Pic(Y)$ such that $ L \cong f^*(L_Y)$? If not, does this hold after possibly replacing $X$ and $Y$ by higher birational models? 

We can see that $f_*(L) \in Pic(Y)$ by Grauert's theorem in case all the fibers are integral, it is easy to show that $L \cong f^*f_*(L)$ (none of this needs nefness). However in general, integrality of all fibers need not be satisfied and I'm not sure how to argue.

In case anyone has any good ideas or references, please let me know.