Let $K$ be a positive integer and $C$ be any $K \times K$ non-singular matrix. For positive real numbers $q_1, \dots, q_K$, define $$\Sigma(q_1, \dots, q_K) = CC' + diag(\frac{1}{q_1}, \dots, \frac{1}{q_K})$$ and $$V(q_1, \dots, q_K) = I_k - C'\Sigma^{-1}C.$$

Question: Are each of the diagonal entries of $V$ log-convex in the arguments $q_1, \dots, q_K$?

The statistical interpretation is the following: suppose a decision maker is learning about $K$ unknowns $\theta_1, \dots, \theta_K$, which are i.i.d. normally distributed according to the prior. She has access to $K$ signals $X_1, \dots, X_K$, which are linear combinations of the unknowns (with coefficients $C$) plus i.i.d. noise. Then $V(q_1, \dots, q_K)$ represents her conditional covariance matrix given $q_k$ observations of signal $X_k$.

It's not too difficult to show that $V_{ii}$ is convex. By brute-force computation I found that it is in fact log-convex when $K = 2$. Thus I'm wondering whether that might be true in general (and why).


It seems that a stronger claim holds. Let $D$ can be any positive definite matrix (not just a diagonal).

The $i$-th entry of the matrix in the OP is \begin{equation*} v_{ii}(D) = 1 - e_i^TC'(CC' + D^{-1})^{-1}Ce_i. \end{equation*}

Key idea. Notice that we can write $v_{ii}(D)$ as the ratio \begin{equation*} v_{ii} = \frac{\det(CC'+D^{-1} - c_ic_i^T)}{\det(CC'+D^{-1})} = \frac{\det(A+D^{-1})}{\det(B+D^{-1})} = \frac{\det(I+AD)}{\det(I+BD)}, \end{equation*} where $B> A \ge 0$.

We wish prove that $v_{ii}$ is log-convex in $D$. Even though $A$ may not invertible, the calculus gets considerably simplified if we assume it is. So we perturb it by a small amount, say $A_{\epsilon} \gets A + \epsilon I$ and perform the argument in terms of $A_{\epsilon}$; then we can conclude the general case by a limiting argument (this should be made rigorous). We abuse notation a bit and omit the subscript on $A$ below for clarity.

\begin{equation*} \log v_{ii}(D) = \log\frac{\det(I+AD)}{\det(I+BD)} =\log\frac{\det(I+A^{1/2}DA^{1/2})}{\det(I+B^{1/2}DB^{1/2})}, \end{equation*} where the latter equivalence follows after similarity transforms on the numerator and denominator.

Computing the first-derivative wrt $D$ we obtain \begin{equation*} \nabla\log v(D) = A^{1/2}(I+A^{1/2}DA^{1/2})^{-1}A^{1/2} - B^{1/2}(I+B^{1/2}DB^{1/2})^{-1}B^{1/2}, \end{equation*} which simplifies to \begin{equation*} \nabla\log v(D) = (A^{-1}+D)^{-1} - (B^{-1}+D)^{-1}. \end{equation*} Now computing second derivatives wrt we see that the Hessian can be identified with the operator \begin{equation*} (B^{-1}+D)^{-1}\otimes (B^{-1}+D)^{-1} - (A^{-1}+D)^{-1} \otimes (A^{-1}+D)^{-1} \ge 0. \end{equation*} The final inequality follows because $A \le B \implies A^{-1}+D\ge B^{-1}+D \implies (A^{-1}+D)^{-1} \le (B^{-1}+D)^{-1}$, combined with the fact that Kronecker products preserve matrix monotonicity.

  • $\begingroup$ To avoid the "\epsilon" argument above, alternatively, we could instead work with $D$ instead of $D^{-1}$ and establish log-concavity in $D$, from which log-convexity in $D$ should follows; I have not verified these details though. $\endgroup$
    – Suvrit
    Jan 24 '17 at 7:45
  • $\begingroup$ That's a very neat idea. I was able to follow your argument until you identify the Hessian matrix with the Kronecker product. What I got is that $$\frac{\partial^2 \det(A+ID)}{\partial D_{ij} \partial D_{i'j'}} = (A^{-1}+D)^{-1}\mid_{j'i} \cdot (A^{-1}+D)^{-1}\mid_{ji'},$$ which doesn't seem to correspond to the Kronecker product. Am I missing something obvious? $\endgroup$ Jan 24 '17 at 16:57
  • $\begingroup$ It does seem true that if we restrict attention to diagonal entries of D, then the corresponding (smaller) Hessian matrix is the Hadamard square of $A^{−1}+D$, which is a principal minor of the Kronecker product. So your argument at least resolves my original question :) $\endgroup$ Jan 24 '17 at 17:16
  • $\begingroup$ Try doing this: $d^2/dt^2 F(D+tH)$, where $F(D)$ is the difference of log-dets up there; you'll obtain something along the lines I mentioned; have a look at mathoverflow.net/questions/214908/… for instance...; in particular because of how we identify the derivative of the inverse of a matrix -- I just differentiated the first derivative wrt to the matrix $D$ to obtain the hessian actually... $\endgroup$
    – Suvrit
    Jan 24 '17 at 18:35
  • $\begingroup$ If I'm not mistaken, $$d^2/dt^2 F(I+A(D+tH)) = -Tr[ ((A^{-1}+D+tH)^{-1} H)^2].$$ Then we need to show $$Tr[ ((A^{-1}+D)^{-1} H)^2] \leq Tr[ ((B^{-1}+D)^{-1} H)^2].$$ How would you show this for general $H$? $\endgroup$ Jan 24 '17 at 21:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.