Are all vector bundles on a given vector bundle the pull back of a vector bundle on the base?

In more detail: let $X$ be a space and $p:E\rightarrow X$ a vector bundle over $X$. Let $\iota: X \rightarrow E$ denote the inclusion given by the zero section. Then for $V$ a vector bundle over $E$, we have the restriction $\iota^* V$ a vector bundle over $X$, while for $W$ vector bundle over $X$ we can consider the pull back $p^*V$ to $E$. On one hand, it is straightforward that $\iota^* p^* W = W$.

When is it true that $p^* \iota^* V$ is isomorphic to $V$? (so that any vector bundle on $E$ is the pull back of a vector bundle on the base)

Clearly the question depends on which setting we are in.

For $X$ a topological space and $E$ a topological vector bundle this is clearly true, since $\iota\circ p$ is homotopic to the identity in $E$. This can be shown using the classifying space for vector bundles and the fact that homotopic maps give the same vector bundle.

For $X$ a manifold and $E$ a smooth vector bundle this is true as well, using the fact that they are isomorphic at the topological level.

What happens in the holomorphic and algebraic setting? Say $X$ a complex manifold, $E$ a holomorphic vector bundle and considering the vector bundles $V$ and $p^*\iota^* V$ up to equivalence of holomorphic vector bundles. Or $X$ a smooth algebraic variety (over $\mathbb{C}$ if needed), $E$ and algebraic vector bundle and equivalence of algebraic vector bundles.

I thought it had to be true, but then I realized that for $X$ equal to a point (in the algebraic setting) this is Quillen-Suslin theorem, so that one need to be more cautious...

Another remark: for $V$ of rank one this is true in both complex and algebraic setting, since there is an isomorphism between the Chow groups of $X$ and $E$.