Let $X$ be a smooth projective variety, and $D\subset X$ a smooth nef and big divisor. Assume that $X$ and $D$ have the same Picard number. 

Under which hypothesis on $D$ may we conclude that there exists an isomorphism $Pic(X)\rightarrow Pic(D)$ over $\mathbb{Z}$ ?