Let $S_n^{++}(\mathbb{R})$ be the space of $n \times n$ symmetric positive definite matrices. For $M \in S_n^{++}(\mathbb{R})$ consider the function $f: X \in S_n^{++}(\mathbb{R}) \mapsto M X^{-1} M$.
My question is the following: can we write $f$ as the natural gradient of a function $\phi: \mathbb{R}^{n \times n} \rightarrow \mathbb{R}$ ? (i.e. can we find $\phi$ such that $f = \nabla \phi$)
This is true if $M=mI_n$ for some $m>0$, and false otherwise.
The function $f$ is a gradient if its differential is symmetric, that is if $$\langle \nabla_Yf(X),Z\rangle=\langle \nabla_Zf(X),Y\rangle$$ for every $X\in S_n^{++}$ and $Y,Z\in S_n$. Because of $\nabla_Yf(X)=-MX^{-1}YX^{-1}M$, this amounts to $${\rm Tr}(MX^{-1}YX^{-1}MZ)={\rm Tr}(MX^{-1}ZX^{-1}MY).$$ With the cyclic property of the trace, and the fact that $\langle A,B\rangle={\rm Tr}(AB)$ is a scalar product, this amounts to $$X^{-1}MZMX^{-1}\equiv MX^{-1}ZX^{-1}M.$$ This rewrites as $B^TZB=Z$ for every $Z$, where $B=MX^{-1}M^{-1}X$. This tells us that $B=I_n$. This being true for every $X\in S_n^{++}$, we obtain the necessary and sufficient condition that $M$ commuttes with every $X^{-1}$. In other words, $M=mI_n$ for some $m>0$.
What is $\phi$ when $M=I_n$ ? Nothing but $\log\det X$. As a matter of fact $$\nabla g(\det X)=g'(\det X)\nabla\det X=g'(\det X)\hat X=g'(\det X)(\det X)X^{-1},$$ thus it suffices to have $g'(s)=1/s$.
