Let $X$ be a smooth projective variety, and $D\subset X$ a smooth nef and big divisor. Assume that $X$ and $D$ have the same Picard number.

Under which hypothesis on $D$ may we conclude that there exists an isomorphism $\mathrm{Pic}(X)\rightarrow \mathrm{Pic}(D)$ over $\mathbb{Z}$ ?