Derivative of pseudoinverse with respect to original matrix

Question

I have been trying to find an analytical expression for the following:

$\frac{\partial {X^{+}}}{\partial {X}}$

In my case, $X$ has a constant rank. I've found the formula for differentiating a pseudoinverse in Golub's paper (equation 4.12):

$$ \frac{\mathrm d}{\mathrm d x} A^+(x) = -A^+ \left( \frac{\mathrm d}{\mathrm d x} A \right) A^+ +A^+ A{^+}^T \left( \frac{\mathrm d}{\mathrm d x} A^T \right) (1-A A^+) + (1-A^+ A) \left( \frac{\mathrm d}{\mathrm d x} A^T \right) A{^+}^T A^+ $$

but I can't see how to input the original matrix.

Answer 1

but I can't see how to input the original matrix.

I think when people write $\tfrac {\partial f} { \partial X}$ or $\tfrac {df}{dX}$ they really mean to find the Fréchet derivative, denoted by $df$, or $Df$. Your formula becomes (appears also in this answer) $$\mathrm d A^+ = -A^+ \left( \mathrm d A \right) A^+ +A^+ A{^+}^T \left( \mathrm d A^T \right) (1-A A^+) + (1-A^+ A) \left( \mathrm d A^T \right) A{^+}^T A^+$$

Since your differentiation is with respect to $A$ itself as a variable, then no chain rule needed, $dA=id$ and the derivative (at the given element $A$) is a linear map $H\mapsto [dA^+](H)$:

$$[\mathrm d A^+](H) = -A^+ \left( H \right) A^+ +A^+ A{^+}^T \left( H^T \right) (1-A A^+) + (1-A^+ A) \left( H^T \right) A{^+}^T A^+$$

Answer 2

Pseudo-inverse is differentiable when the underlying matrix has a constant rank. An explicit formula is then available. This is the classical reference on the subject: http://epubs.siam.org/doi/abs/10.1137/0710036