Suppose we have a typical logdet function $\mathcal{L}$ $$ \mathcal{L} = \log\vert \mathbf{I} + \mathbf{A}\mathbf{S} \vert - \mathbf{q}^T(\mathbf{A}^{-1} + \mathbf{S})^{-1} \mathbf{q}, $$ where $\mathbf{S}$ is a Symmetric Positive Semi-Definite Matrix, $\mathbf{q}$ is a column vector, $\mathbf{I}$ is an identity matrix. The local minima of $\mathcal{L}$ w.r.t $\mathbf{A}$ could be found by solving $\mathbf{A}$ ($\frac{\partial \mathcal{L}}{\partial \mathbf{A}} = \mathbf{0}$), we get the closed form solution of $\mathbf{A}$ $$ \mathbf{A} = \mathbf{S}^{-1}(\mathbf{S} - \mathbf{q}\mathbf{q}^T)\mathbf{S}^{-1} $$

The problem is, what if I want to constrain the descent path of the logdet function by imposing extra structures on $\mathbf{A}$ ? For example,

  1. $\mathbf{A}$ is p.s.d and have a simple structure, which constrains that at each step, $\mathbf{A}$ should be: $\mathbf{A}=r\mathbf{I}$.

  2. or we constrain that $\mathbf{A}$ is p.s.d and have a band-p Symmetric Toeplitz structure, i.e., \begin{equation} \mathbf{A} = r \begin{pmatrix} 1 & p & 0 & 0 \\\ p & 1 & p & 0 \\\ 0 & p & 1 & p \\\ 0 & 0 & p & 1 \end{pmatrix}. \end{equation}

How could I analyze the saddle point of $\mathcal{L}$ with these constrains ? Will these constrains affect achieving the local minima of $\mathcal{L}$ and provide a more robust result that more likely to find the true minimal that has this type of structure ?


@liubenyuan, some remarks. Necessarily $S>0$. On the other hand, your result is correct up a signum (have a look at the case $n=1$); infortunately that makes a big difference ! The derivative is $0$ if $A=S^{-1}(qq^T-S)S^{-1}$ and you want that $A\geq 0$, that is $qq^T\geq S$. Let $(\lambda_i\geq 0)_i$ be the spectrum of $S$. According to the min-max theorem $||q||^2\geq \lambda_1$ and, for every $i>1$, $\lambda_i=0$ and $S$ cannot be invertible except if $n=1$. Finally, there are no critical points, except for $n=1$, $a=q^2/s^2-1/s$ with $q^2\geq s$.

EDIT: cf. the following post. Have you read my answer to this post ?

Hessian of function of covariance matrices

  • $\begingroup$ Hi, sorry for the late reply, I am on a training session for about half a year. I will read your post ASAP. $\endgroup$ – liubenyuan Apr 2 '14 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.