If $\mathbf{P}^1$ is replaced by the affine line $\mathbf{A}^1$, this becomes the cancellation problem, and we have a pair of famous Danielewski surfaces ($xy=1-z^2$ and $x^2y=1-z^2$) as a counterexample (though I'm still seeking how to prove that..). I suppose in my case this counterexample might no longer work.
Also one may replace $\mathbf{P}^1$ by $\mathbf{P}^n$ or other fixed varieties, or ask again after imposing some conditions on $V$ and $W$ (for example dimensions) if there are counterexamples for my question. And more wildly I may ask for what kind of family $X_n$, we will have the result that $V\times X_n=W\times X_n$ implies $V=W$. Any result of these kind of variations of the problem is also welcomed.