MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm trying to find a expression for the matrix derivative with respect to the pseudo-inverse of a matrix. So, i have some function $f(A)$ of a matrix $A$, which is singular. If it weren't I could use that $$ \frac{df(A)}{dA^{-1}} = -A^{-1}\frac{df(A)}{dA}A^{-1}, $$ but I can't right? So does anyone know where I could find a pseudo-inverse version of this? So basically I want an expression for $\frac{df(A)}{dA^+}$, and yes I reckon it won't be as cleas and simple as the one above. Also, does anyone know where I could find pseudo-inverse generalizations of all those classic matrix inversion lemmas?

Thanks in advance for any comments!

share|cite|improve this question
The fonts render terribly, but there appears to be a derivation in the 'Mathematica Journal':… – Steven Pav Jun 25 '14 at 4:22

To address your second question, here are two useful references:

  1. An Extension of the Matrix Inversion Lemma by Nariyasu Minamide in SIAM J. Alg. and Disc. Methods, 6, pp. 371-377 (1985).
  2. The Moore-Penrose generalized inverse for sums of matrices by J. A. Fill and D. E. Fishkind. (1998)
share|cite|improve this answer
Cool, thanks for the links. This seems to answer the second question pretty well! – alexsuse Feb 7 '12 at 13:55

@ alexsuse , your formula is false. To see that, choose $u(A)=A^{-1}$.

You want the derivative of the function $g:A^{-1}\rightarrow A\rightarrow f(A)$, that is $g=f\circ u$ with $Du_{A^{-1}}:H\rightarrow -AHA$. Thus $Dg_{A^{-1}}:H\in M_n\rightarrow Df_A(-AHA)$. In other words, if $A^{-1}$ is a function of $t$, then $\dfrac{\partial{g}}{\partial{t}}=Df_A(-A\dfrac{\partial{A^{-1}}}{\partial{t}} A)$.

EDIT: let $t\rightarrow A(t)$ be a real analytic function (I don't know if this is a necessary condition) s.t. $A(0)$ is singular and, for every $t\not=0$ (in a neighborhood of $0$), $A(t)$ is not singular. Then it seems to me that my last formula works in the form $\dfrac{\partial{g}}{\partial{t}}(0)=\lim_{t\rightarrow 0}Df_A(-A\dfrac{\partial{A^{-1}(t)}}{\partial{t}} A)$. See this example: $f(A)=A(t)=\begin{pmatrix}-3t^2+t&5t^2-t\\-6t^2+4t&-t^2+6t\end{pmatrix}$. Yet, if $A(t)$ is singular, then my last formula (replacing $A^{-1}$ with $A^+$) does not work; for instance: $f(A)=A(t)=\begin{pmatrix}2&3\\4+2t&6+3t\end{pmatrix}$. Instead of $\dfrac{\partial{g}}{\partial{t}}(0)=\begin{pmatrix}0&0\\2&3\end{pmatrix}$ we obtain $\dfrac{\partial{g}}{\partial{t}}(0)=\begin{pmatrix}4/5&6/5\\8/5&12/5\end{pmatrix}$.

EDIT: the inverse function $u$ was mixed up with $f$ originally.

share|cite|improve this answer
Not sure what you meant. What is $u$ in your notation, and what do you mean by the $Du_{A^{-1}}:H$ notation? Note that I'm considering the derivative of a matrix with respect to another matrix. In that sense, $df/dA$ is a matrix ($(df/dA)_{ij} = \frac{\partial f}{\partial A_{ij}}$), and so is $df/d(A^{-1})$. In your example, I think you are considering a matrix which is a function of a parameter and looking at the derivative with respect to that parameter. – alexsuse Sep 12 '14 at 10:57
@ alexsuse, $Du_{A^{-1}}$ is the derivative of $u:X\rightarrow X^{-1}$ in $A^{-1}$; it is a linear application that is defined on $M_n(\mathbb{R})$ with values in $M_n(\mathbb{R})$. The derivative of the function composition is $Dg_{A^{-1}}=Df_A\circ Du_{A^{-1}}$. My last formula is obtained when we compose with any function $t\rightarrow A^{-1}(t)$. Instead of reading the Matrix Cookbook, read a good book about differential calculus. – loup blanc Sep 12 '14 at 11:36
Such snark, much calculus :D. FWIW, I think you meant $u(A) = A^{-1}$ in the first line. At least it makes more sense that way. And technically speaking, you didn't show that my formula is false. My formula is written in terms of $A^{-1}$, so it is only natural that it would not hold if $A$ is not invertible (that is actually part of my question). That being said, thanks for the comments. – alexsuse Feb 3 '15 at 17:49

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.