@ alexsuse , your formula is false. To see that, choose $f(A)=A^{-1}$.

You want the derivative of the function $g:A^{-1}\rightarrow A\rightarrow f(A)$, that is $g=f\circ u$ with $Du_{A^{-1}}:H\rightarrow -AHA$. Thus $Dg_{A^{-1}}:H\in M_n\rightarrow Df_A(-AHA)$. In other words, if $A^{-1}$ is a function of $t$, then $\dfrac{\partial{g}}{\partial{t}}=Df_A(-A\dfrac{\partial{A^{-1}}}{\partial{t}} A)$.

EDIT: let $t\rightarrow A(t)$ be a real analytic function (I don't know if this is a necessary condition) s.t. $A(0)$ is singular and, for every $t\not=0$ (in a neighborhood of $0$), $A(t)$ is not singular. Then it seems to me that my last formula works in the form $\dfrac{\partial{g}}{\partial{t}}(0)=\lim_{t\rightarrow 0}Df_A(-A\dfrac{\partial{A^{-1}(t)}}{\partial{t}} A)$. See this example: $f(A)=A(t)=\begin{pmatrix}-3t^2+t&5t^2-t\\-6t^2+4t&-t^2+6t\end{pmatrix}$. Yet, if $A(t)$ is singular, then my last formula (replacing $A^{-1}$ with $A^+$) does not work; for instance: $f(A)=A(t)=\begin{pmatrix}2&3\\4+2t&6+3t\end{pmatrix}$. Instead of $\dfrac{\partial{g}}{\partial{t}}(0)=\begin{pmatrix}0&0\\2&3\end{pmatrix}$ we obtain $\dfrac{\partial{g}}{\partial{t}}(0)=\begin{pmatrix}4/5&6/5\\8/5&12/5\end{pmatrix}$.

@ alexsuse , your formula is false. To see that, choose $u(A)=A^{-1}$.

You want the derivative of the function $g:A^{-1}\rightarrow A\rightarrow f(A)$, that is $g=f\circ u$ with $Du_{A^{-1}}:H\rightarrow -AHA$. Thus $Dg_{A^{-1}}:H\in M_n\rightarrow Df_A(-AHA)$. In other words, if $A^{-1}$ is a function of $t$, then $\dfrac{\partial{g}}{\partial{t}}=Df_A(-A\dfrac{\partial{A^{-1}}}{\partial{t}} A)$.

EDIT: let $t\rightarrow A(t)$ be a real analytic function (I don't know if this is a necessary condition) s.t. $A(0)$ is singular and, for every $t\not=0$ (in a neighborhood of $0$), $A(t)$ is not singular. Then it seems to me that my last formula works in the form $\dfrac{\partial{g}}{\partial{t}}(0)=\lim_{t\rightarrow 0}Df_A(-A\dfrac{\partial{A^{-1}(t)}}{\partial{t}} A)$. See this example: $f(A)=A(t)=\begin{pmatrix}-3t^2+t&5t^2-t\\-6t^2+4t&-t^2+6t\end{pmatrix}$. Yet, if $A(t)$ is singular, then my last formula (replacing $A^{-1}$ with $A^+$) does not work; for instance: $f(A)=A(t)=\begin{pmatrix}2&3\\4+2t&6+3t\end{pmatrix}$. Instead of $\dfrac{\partial{g}}{\partial{t}}(0)=\begin{pmatrix}0&0\\2&3\end{pmatrix}$ we obtain $\dfrac{\partial{g}}{\partial{t}}(0)=\begin{pmatrix}4/5&6/5\\8/5&12/5\end{pmatrix}$.

EDIT: the inverse function $u$ was mixed up with $f$ originally.

@ alexsuse , your formula is false. To see that, choose $f(A)=A^{-1}$.

You want the derivative of the function $g:A^{-1}\rightarrow A\rightarrow f(A)$, that is $g=f\circ u$ with $Du_{A^{-1}}:H\rightarrow -AHA$. Thus $Dg_{A^{-1}}:H\in M_n\rightarrow Df_A(-AHA)$. In other words, if $A^{-1}$ is a function of $t$, then $\dfrac{\partial{g}}{\partial{t}}=Df_A(-A\dfrac{\partial{A^{-1}}}{\partial{t}} A)$.

@ alexsuse , your formula is false. To see that, choose $f(A)=A^{-1}$.

You want the derivative of the function $g:A^{-1}\rightarrow A\rightarrow f(A)$, that is $g=f\circ u$ with $Du_{A^{-1}}:H\rightarrow -AHA$. Thus $Dg_{A^{-1}}:H\in M_n\rightarrow Df_A(-AHA)$. In other words, if $A^{-1}$ is a function of $t$, then $\dfrac{\partial{g}}{\partial{t}}=Df_A(-A\dfrac{\partial{A^{-1}}}{\partial{t}} A)$.

EDIT: let $t\rightarrow A(t)$ be a real analytic function (I don't know if this is a necessary condition) s.t. $A(0)$ is singular and, for every $t\not=0$ (in a neighborhood of $0$), $A(t)$ is not singular. Then it seems to me that my last formula works in the form $\dfrac{\partial{g}}{\partial{t}}(0)=\lim_{t\rightarrow 0}Df_A(-A\dfrac{\partial{A^{-1}(t)}}{\partial{t}} A)$. See this example: $f(A)=A(t)=\begin{pmatrix}-3t^2+t&5t^2-t\\-6t^2+4t&-t^2+6t\end{pmatrix}$. Yet, if $A(t)$ is singular, then my last formula (replacing $A^{-1}$ with $A^+$) does not work; for instance: $f(A)=A(t)=\begin{pmatrix}2&3\\4+2t&6+3t\end{pmatrix}$. Instead of $\dfrac{\partial{g}}{\partial{t}}(0)=\begin{pmatrix}0&0\\2&3\end{pmatrix}$ we obtain $\dfrac{\partial{g}}{\partial{t}}(0)=\begin{pmatrix}4/5&6/5\\8/5&12/5\end{pmatrix}$.