1
$\begingroup$

Let $M$ be an SPD matrix and let $\Pi=QQ^T$ be the orthogonal projection onto the range of $Q$ (a "tall" matrix with orthonormal columns). I have an expression in the form $$\tag{1} K=\max_{v}\frac{v^T(I-\Pi)v}{v^T(I-\Pi)M^{-1}(I-\Pi)v}, $$ which I would like to express as a maximum of something which has $M$ in the numerator and such that neither the denominator nor the (sub)space over which the maximisation is made depend on $M$.

Actually I know that the solution should be $$\tag{2} K=\max_v\frac{v^T(I-\tilde{\Pi})^TM(I-\tilde{\Pi})v}{v^T(I-\Pi)v}, $$ where $\tilde{\Pi}=Q(Q^TMQ)^{-1}Q^TM$ (this pie is the $M$-orthogonal projection onto the range of $Q$). I would appreciate any hint on how to get from (1) to (2) (or vice versa).


I've tried to get from (2) to (1). First, $$(I-\tilde{\Pi})^TM(I-\tilde{\Pi})=M^{1/2}(I-\overline{\Pi})M^{1/2},$$ where now $$\overline{\Pi}=M^{1/2}Q(Q^TMQ)^{-1}Q^TM^{1/2}$$ is the orthogonal projection onto the range of $M^{1/2}Q$. So putting this to (2) and taking $w=M^{1/2}v$ gives $$ K=\max_w\frac{w^T(I-\overline{\Pi})w}{w^TM^{-1/2}(I-\Pi)M^{-1/2}w}, $$ which has already a sort of $M^{-1}$ in the denominator, but still not in the form of (1) and with a different projector in the numerator.


P.S.: I've asked a related question already, but the simply stated solution there does not much help me to get to (2) as the subspace over which the maximum is evaluated depends on $M$. What I'm actually looking for is a characterisation of (1) which does not change the subspace over which we maximise. In fact, (1) and (2) give that $$ K=\max_{w\in\mathcal{R}(I-\Pi)}\frac{w^Tw}{w^TM^{-1}w}=\max_{w\in\mathcal{R}(I-\Pi)}\frac{w^T(I-\tilde{\Pi})^TM(I-\tilde{\Pi})w}{w^Tw}. $$

$\endgroup$

1 Answer 1

0
$\begingroup$

After some playing with this problem, I think I've found a solution.

Let $U$ be an orthonormal basis of the range of $I-\Pi$, that is, $[Q,U]$ is a square orthogonal matrix such that $U^TQ=0$ and $I-\Pi=UU^T$. Then $$ \begin{split} K&=\max_v\frac{v^T(I-\Pi)v}{v^T(I-\Pi)M(I-\Pi)v}=\max_v\frac{v^TUU^Tv}{v^TUU^TM^{-1}UU^Tv} =\max_v\frac{v^T(U^TM^{-1}U)^{-1}v}{v^Tv}\\ &=\max_v\frac{v^T(U^TM^{-1}U)^{-1}U^TM^{-1}U(U^TM^{-1}U)^{-1}v}{v^Tv}\\ &=\max_v\frac{v^TU^TU(U^TM^{-1}U)^{-1}U^TM^{-1}MM^{-1}U(U^TM^{-1}U)^{-1}U^TUv}{v^TU^TUv}\\ &=\max_{v\in\mathcal{R}(I-\Pi)}\frac{v^T[U(U^TM^{-1}U)^{-1}U^TM^{-1}]M[M^{-1}U(U^TM^{-1}U)^{-1}U^T]v}{v^Tv}. \end{split} $$ What remains to show is that $$\tag{1}I-\tilde{\Pi}=I-Q(Q^TMQ)^{-1}Q^T\quad\text{and}\quad\Phi=M^{-1}U(U^TM^{-1}U)^{-1}U^T$$ are equal. It is easy to show that the square matrix $[U,MQ]$ is nonsingular. So $I-\Pi=\Phi$ if and only if $[U,MQ]^T(I-\Pi-\Phi)=0$. But this is easy to verify using $U^TQ=0$ and the expressions in (1).

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .