If your manifold is complex projective, then the answer is yes. Otherwise it is no. You can take a $K3$ surface without complex curves and just consider its tangent bundle. Of curse it will stay holomorphically non-trivial, if you throw away finite number of points from $K3$, since any holomorphic vector field on a $K3$ surface defined outside a finite set is zero.
On the other hand, I guess, in this example the bundle is topoligcally trivial, since its first Chern class is zero.
Added. Let me sketch the proof of the statement, everything holds for compact complex aglebraic manifold. First every such a manifold admits a blow up that is projective. Pull back the complex bundle to the blow up. Then we get a holomorphic bundle over a complex projective manifold and such a bundle is algebraic. Hence it has meromorphic sections. Moreover we can chose meromorphic sections that a linearly independent at one point. It is clear that they trivialise the bundle over a complement to a complex analytic manifold.
