MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose we have a typical logdet function $\mathcal{L}$ $$ \mathcal{L} = \log\vert \mathbf{I} + \mathbf{A}\mathbf{S} \vert - \mathbf{q}^T(\mathbf{A}^{-1} + \mathbf{S})^{-1} \mathbf{q}, $$ where $\mathbf{S}$ is a Symmetric Positive Semi-Definite Matrix, $\mathbf{q}$ is a column vector, $\mathbf{I}$ is an identity matrix. The local minima of $\mathcal{L}$ w.r.t $\mathbf{A}$ could be found by solving $\mathbf{A}$ ($\frac{\partial \mathcal{L}}{\partial \mathbf{A}} = \mathbf{0}$), we get the closed form solution of $\mathbf{A}$ $$ \mathbf{A} = \mathbf{S}^{-1}(\mathbf{S} - \mathbf{q}\mathbf{q}^T)\mathbf{S}^{-1} $$

The problem is, what if I want to constrain the descent path of the logdet function by imposing extra structures on $\mathbf{A}$ ? For example,

  1. $\mathbf{A}$ is p.s.d and have a simple structure, which constrains that at each step, $\mathbf{A}$ should be: $\mathbf{A}=r\mathbf{I}$.

  2. or we constrain that $\mathbf{A}$ is p.s.d and have a band-p Symmetric Toeplitz structure, i.e., \begin{equation} \mathbf{A} = r \begin{pmatrix} 1 & p & 0 & 0 \\\ p & 1 & p & 0 \\\ 0 & p & 1 & p \\\ 0 & 0 & p & 1 \end{pmatrix}. \end{equation}

How could I analyze the saddle point of $\mathcal{L}$ with these constrains ? Will these constrains affect achieving the local minima of $\mathcal{L}$ and provide a more robust result that more likely to find the true minimal that has this type of structure ?

share|cite|improve this question
up vote 0 down vote accepted

@liubenyuan, some remarks. Necessarily $S>0$. On the other hand, your result is correct up a signum (have a look at the case $n=1$); infortunately that makes a big difference ! The derivative is $0$ if $A=S^{-1}(qq^T-S)S^{-1}$ and you want that $A\geq 0$, that is $qq^T\geq S$. Let $(\lambda_i\geq 0)_i$ be the spectrum of $S$. According to the min-max theorem $||q||^2\geq \lambda_1$ and, for every $i>1$, $\lambda_i=0$ and $S$ cannot be invertible except if $n=1$. Finally, there are no critical points, except for $n=1$, $a=q^2/s^2-1/s$ with $q^2\geq s$.

EDIT: cf. the following post. Have you read my answer to this post ?

Hessian of function of covariance matrices

share|cite|improve this answer
Hi, sorry for the late reply, I am on a training session for about half a year. I will read your post ASAP. – liubenyuan Apr 2 '14 at 12:57

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.