Yes.
Let $U=\{A\in B(X,Y);\text{there is }B\in B(Y,X)\text{ so that }BA=I_X\}$.
(You probably want to have the left inverse in $B(Y,X)$. Otherwise you can get counterexamples from compact injections.)

Take any $A\in U$.
Then there is $B\in B(Y,X)$ so that $BA=I_X$.
Suppose $C\in B(X,Y)$ satisfies $\|C\|\leq\frac12\|B\|^{-1}$.
Then $B(A+C)=I_X+BC\in B(X,X)$.
Now $\|BC\|\leq\|B\|\cdot\|C\|\leq\frac12$, so $I_X+BC$ has a continuous inverse by Neumann series.
Since $(I_X+BC)^{-1}B(A+C)=I_X$, the operator $A+C$ has a continuous left inverse.
Therefore $A$ is an interior point of $U$.