# Log-convexity of conditional variances

Let $K$ be a positive integer and $C$ be any $K \times K$ non-singular matrix. For positive real numbers $q_1, \dots, q_K$, define $$\Sigma(q_1, \dots, q_K) = CC' + diag(\frac{1}{q_1}, \dots, \frac{1}{q_K})$$ and $$V(q_1, \dots, q_K) = I_k - C'\Sigma^{-1}C.$$

Question: Are each of the diagonal entries of $V$ log-convex in the arguments $q_1, \dots, q_K$?

The statistical interpretation is the following: suppose a decision maker is learning about $K$ unknowns $\theta_1, \dots, \theta_K$, which are i.i.d. normally distributed according to the prior. She has access to $K$ signals $X_1, \dots, X_K$, which are linear combinations of the unknowns (with coefficients $C$) plus i.i.d. noise. Then $V(q_1, \dots, q_K)$ represents her conditional covariance matrix given $q_k$ observations of signal $X_k$.

It's not too difficult to show that $V_{ii}$ is convex. By brute-force computation I found that it is in fact log-convex when $K = 2$. Thus I'm wondering whether that might be true in general (and why).

It seems that a stronger claim holds. Let $D$ can be any positive definite matrix (not just a diagonal).

The $i$-th entry of the matrix in the OP is \begin{equation*} v_{ii}(D) = 1 - e_i^TC'(CC' + D^{-1})^{-1}Ce_i. \end{equation*}

Key idea. Notice that we can write $v_{ii}(D)$ as the ratio \begin{equation*} v_{ii} = \frac{\det(CC'+D^{-1} - c_ic_i^T)}{\det(CC'+D^{-1})} = \frac{\det(A+D^{-1})}{\det(B+D^{-1})} = \frac{\det(I+AD)}{\det(I+BD)}, \end{equation*} where $B> A \ge 0$.

We wish prove that $v_{ii}$ is log-convex in $D$. Even though $A$ may not invertible, the calculus gets considerably simplified if we assume it is. So we perturb it by a small amount, say $A_{\epsilon} \gets A + \epsilon I$ and perform the argument in terms of $A_{\epsilon}$; then we can conclude the general case by a limiting argument (this should be made rigorous). We abuse notation a bit and omit the subscript on $A$ below for clarity.

\begin{equation*} \log v_{ii}(D) = \log\frac{\det(I+AD)}{\det(I+BD)} =\log\frac{\det(I+A^{1/2}DA^{1/2})}{\det(I+B^{1/2}DB^{1/2})}, \end{equation*} where the latter equivalence follows after similarity transforms on the numerator and denominator.

Computing the first-derivative wrt $D$ we obtain \begin{equation*} \nabla\log v(D) = A^{1/2}(I+A^{1/2}DA^{1/2})^{-1}A^{1/2} - B^{1/2}(I+B^{1/2}DB^{1/2})^{-1}B^{1/2}, \end{equation*} which simplifies to \begin{equation*} \nabla\log v(D) = (A^{-1}+D)^{-1} - (B^{-1}+D)^{-1}. \end{equation*} Now computing second derivatives wrt we see that the Hessian can be identified with the operator \begin{equation*} (B^{-1}+D)^{-1}\otimes (B^{-1}+D)^{-1} - (A^{-1}+D)^{-1} \otimes (A^{-1}+D)^{-1} \ge 0. \end{equation*} The final inequality follows because $A \le B \implies A^{-1}+D\ge B^{-1}+D \implies (A^{-1}+D)^{-1} \le (B^{-1}+D)^{-1}$, combined with the fact that Kronecker products preserve matrix monotonicity.

• To avoid the "\epsilon" argument above, alternatively, we could instead work with $D$ instead of $D^{-1}$ and establish log-concavity in $D$, from which log-convexity in $D$ should follows; I have not verified these details though. Jan 24 '17 at 7:45
• That's a very neat idea. I was able to follow your argument until you identify the Hessian matrix with the Kronecker product. What I got is that $$\frac{\partial^2 \det(A+ID)}{\partial D_{ij} \partial D_{i'j'}} = (A^{-1}+D)^{-1}\mid_{j'i} \cdot (A^{-1}+D)^{-1}\mid_{ji'},$$ which doesn't seem to correspond to the Kronecker product. Am I missing something obvious? Jan 24 '17 at 16:57
• It does seem true that if we restrict attention to diagonal entries of D, then the corresponding (smaller) Hessian matrix is the Hadamard square of $A^{−1}+D$, which is a principal minor of the Kronecker product. So your argument at least resolves my original question :) Jan 24 '17 at 17:16
• Try doing this: $d^2/dt^2 F(D+tH)$, where $F(D)$ is the difference of log-dets up there; you'll obtain something along the lines I mentioned; have a look at mathoverflow.net/questions/214908/… for instance...; in particular because of how we identify the derivative of the inverse of a matrix -- I just differentiated the first derivative wrt to the matrix $D$ to obtain the hessian actually... Jan 24 '17 at 18:35
• If I'm not mistaken, $$d^2/dt^2 F(I+A(D+tH)) = -Tr[ ((A^{-1}+D+tH)^{-1} H)^2].$$ Then we need to show $$Tr[ ((A^{-1}+D)^{-1} H)^2] \leq Tr[ ((B^{-1}+D)^{-1} H)^2].$$ How would you show this for general $H$? Jan 24 '17 at 21:17