Here is a (slightly) alternative approach in the smooth case. Assume the embedding $Y \to X$ has codimension at least three, where $Y$ is a closed smooth manifold.
What we need to know is that the inclusion $ X-Y \to X$ is at least $2$-connected. This exercise is an easy consequence of transversality, using the codimension $\ge 3$ hypothesis.
Isomorphism classes of line bundles over $X$ are classified by homotopy classes of maps $X \to \Bbb CP^\infty$. I.e., they're in bijection with $H^2(X;\Bbb Z)$, where the trivial class corresponds to the trivial bundle.
But the map $H^2(X;\Bbb Z) \to H^2(X-Y;\Bbb Z)$ is injective by the Hurewicz theorem and the long exact sequence of a pair (since $X-Y\to X$ is $2$-connected). By hypothesis the class of $L$ in $H^2(X;\Bbb Z)$ pushes forward to a trivial class in $H^2(X-Y;\Bbb Z)$, so it follows that $L$ is trivializable.
(More generally, for a subcomplex $Y \to X$, your conclusion will hold whenever $Y-X\to X$ is $2$-connected.)