Suppose we have a typical logdet function $\mathcal{L}$ with respect to a covariance matrix $\mathbf{A}$,
$$
\mathcal{L}(\mathbf{A}) = \log\vert \mathbf{I} + \mathbf{A}\mathbf{S} \vert - \mathbf{q}^T(\mathbf{A}^{-1} + \mathbf{S})^{-1} \mathbf{q},
$$
where $\mathbf{S}$ is a Symmetric Positive Semi-Definite Matrix, $\mathbf{q}$ is a column vector, $\mathbf{I}$ is an identity matrix.

The **questions** are

  1. How could I calculate the gradient of $\mathcal{L}(\mathbf{A})$, Given that $\mathbf{A}$ is a Symmetric Positive Definite Matrix ? I found in literature that if $\mathbf{A}$ is a Symmetric matrix, $\frac{\partial \log\vert \mathbf{A}\vert}{\partial \mathbf{A}}= \mathbf{A}^{-1} + \mathbf{A}^{-T} - \mathbf{I}\odot\mathbf{A}^{-1}$, where $\odot$ is Hadamard Product. However, I found that most of current literatures simply assumes that $\frac{\partial \log\vert \mathbf{A}\vert}{\partial \mathbf{A}} = \mathbf{A}^{-T}$. Will these difference affects the gradient result?

  2. How could I calculate the Hessian of $\mathcal{L}(\mathbf{A})$? i.e., $H = \frac{\partial \mathcal{L}(\mathbf{A})}{\partial^2\mathbf{A}}$, where I need to calculate the derivatives of a Matrix to a Matrix. If I want to find the minima, maxima, saddle points of $\mathbf{H}$, should the result that the Hessian matrix $\mathbf{H}$ being positive definite, negative definite, and none definite still holds ? How could I find the minima by exploiting the Hessian matrix, which is a matrix-by-matrix derivatives.