Let $X$ be a smooth complex projective variety and $p:Y\to X$ be a smooth $\mathbb CP^k$-bundle (i.e. locally trivial in analytic topology). Suppose that there exists a line bundle $L$ on $Y$ that restricts to $\mathcal O(1)$ on each $\mathbb CP^k$-fibre.

Question. Is it true that there is a very ample line bundle $L'$ on $X$ such that $p^*L'\otimes L$ is very ample on $Y$?

I feel like this should be deducible from Kodaira vanishing, but can't prove so far.