2
$\begingroup$

TLDR; I am trying to prove the existence of an infinitesimal rotation which always moves a matrix "closer" to being orthogonal.

Setting

In this setting, we have a matrix $W \in \mathbb{R}^{n \times m}$ with $n \geq m$. We then define two new matrices, $M = W^TW$ and $D = \text{diag}(M)$. I am concerned primarily with the quantity:

$L(D, M) = \log\det D - \log\det M$

We have $L \geq 0$, which we can derive from the KL divergence between two multivariate Gaussians: $L = KL(\mathcal{N}(0, D^{-1})|| \mathcal{N}(0, M^{-1})) \geq 0$, and $L = 0 \iff M\text{ is diagonal} \iff W \text{ has orthogonal columns}$.

Problem

Now, I want to apply a small rotation to $W$, i.e. $W' = WR$ such that $R^TR = I$. This way, the log determinant of $M' = W'^TW'$ is equal to the log determinant of $M$. However, if $R$ is chosen carefully then the log determinant of $D$ would decrease.

One (non-small) example could be found from the SVD of $W = USV^T$, setting $R=V$ would make $W'$ orthogonal and $M'$ a diagonal matrix.

Finally, my question is: Does there exist an infinitesimal rotation matrix $R$ which reduces $L$?

My attempt so far

We can define an infinitesimal rotation through the Lie algebra of the special orthogonal group. That is, choose an upper triangular matrix $E$, whose entries are all less than $\epsilon$. Then $A = \exp\{E^T - E\}$ is orthogonal, with $A = I + E^T - E + O(\epsilon^2)$. Now I could compute the elements of $D' = \text{diag}(A^TW^TWA)$ up to first order and try to find elements of E which reduce the log determinant. Unfortunately, it has been difficult to prove that such an $E$ exists in general.

Another approach may use surjectivity of the exponential map to say that some matrix $B$ exists such that $\exp\{B^T - B\} = V$, and then $\epsilon B$ probably decreases $L$ for a small enough $\epsilon$?

Is there a simpler way to show that some near-identity orthogonal matrix which decreases $L$ must exist? Perhaps we can use the smooth-manifold structure on orthogonal matrices to argue that in an open neighborhood around the identity there must be some matrix with the properties we want?

As a final comment, this seems related to Wahba's problem or the Orthogonal Procrustes problem but with an additional constraint on the matrix.

Improve this question
$\endgroup$
3
  • $\begingroup$ This is possible using a unitary matrix for $R$: In the SVD of $W$, let $V = e^{iH}$ with $H$ Hermitian; this is possible since $V$ is unitary. Now for any given $\epsilon$ choose $n$ sufficiently large that all the elements $R_{ij} = e^{iH/n}$ are within $\epsilon$ of $\delta_{ij}$. Then your $L$ using $WR$ is always less than or equal to $L$ using $W$, with equality only if they are both zero. However, that does not say how to do this with an orthogonal (not a unitary) matrix. $\endgroup$
    – Mark Fischler
    Commented Mar 25, 2019 at 16:09
  • $\begingroup$ Thanks for the comment! Is it not the case that the same can be done for orthogonal/special orthogonal matrices? They are compact Lie groups and thus the exponential map from the skew-symmetric matrices is surjective. So we can write $V=e^{(B-B^T)/n}$. However, I'm not sure how to prove that this will decrease L for a large enough $L$ (though it is intuitive). Could you elaborate and how you could show this holds (say, in the unitary case you proposed)? $\endgroup$
    – user124784
    Commented Mar 25, 2019 at 16:15
  • $\begingroup$ I see now that as the orthogonal group is not connected the exponential map is not surjective. The special orthogonal group (which is connected) is unlikely to be enough? For my purposes, any infinitesimal orthogonal matrix is sufficient (that is, it need not have +1 determinant). I will edit the question the clarify this. With that said, I am still not sure how one would prove that $L$ decreases even for unitary $R$. $\endgroup$
    – user124784
    Commented Mar 25, 2019 at 17:37

0

Reset to default

You must log in to answer this question.