I formulated a statement, which is hopefully true (at least I'm not knowledgeable enough to see a reason for it not to be). However, I'm struggling to come up with a proof.
Let $W_i \in \mathbb{R}^{d_i \times d_{i - 1}}, \ i = 1, 2, \ d := \min d_i$. Consider $R := W_2 W_1$ with $\textrm{rank}(R) \leq d$.
Can we obtain any matrix close to $R$ and of rank not larger than that of $R$ by small perturbations of the factors $W_j$?
More formally, let $\epsilon>0$ be given. Can we then find a $\delta>0$ such that for any $P$ with $\|P-R\|<\delta$ and $\textrm{rank}(P) \leq d$, there are $V_i$ such that $V_2V_1=P$ and $\|V_i-W_i\|<\epsilon$?
Does the statement seem correct to you? How could I prove it? Thank you.