Let $f:X \to Y$ be a smooth projective morphism between smooth quasi-projective complex varieties such that fibers of $f$ over closed points are connected.
Assume that $S\subset Y$ is a non-empty complement of a union of countably many closed subvarieties of $Y$ such that $Pic(f^{-1}(y))$ is of rank one for any closed point $y\in S$.
Question: Is $Pic(f^{-1}(o))$ also of rank one, where $o\in S\subset Y$ is the generic point of $Y$? And is it true that $\mathrm{rk}(Pic(X))\leq \mathrm{rk}(Pic(Y))+1$?
The example in my mind is when $f$ is a smooth family of abelian varieties.