Yes. Let $U=\{A\in B(X,Y);\text{there is }B\in B(Y,X)\text{ so that }BA=I_X\}$. (You probably want to have the left inverse in $B(Y,X)$. Otherwise you can get counterexamples from compact injections.)
Take any $A\in U$. Then there is $B\in B(Y,X)$ so that $BA=I_X$. Suppose $C\in B(X,Y)$ satisfies $\|C\|\leq\frac12\|B\|^{-1}$. Then $B(A+C)=I_X+BC\in B(X,X)$. Now $\|BC\|\leq\|B\|\cdot\|C\|\leq\frac12$, so $I_X+BC$ has a continuous inverse by Neumann series. Since $(I_X+BC)^{-1}B(A+C)=I_X$, the operator $A+C$ has a continuous left inverse. Therefore $A$ is an interior point of $U$.