Let $M$ be an SPD matrix and let $\Pi=QQ^T$ be the orthogonal projection onto the range of $Q$ (a "tall" matrix with orthonormal columns). I have an expression in the form $$\tag{1} K=\max_{v}\frac{v^T(I-\Pi)v}{v^T(I-\Pi)M^{-1}(I-\Pi)v}, $$ which I would like to express as a maximum of something which has $M$ in the numerator and such that neither the denominator nor the (sub)space over which the maximisation is made depend on $M$.

Actually I know that the solution should be $$\tag{2} K=\max_v\frac{v^T(I-\tilde{\Pi})^TM(I-\tilde{\Pi})v}{v^T(I-\Pi)v}, $$ where $\tilde{\Pi}=Q(Q^TMQ)^{-1}Q^TM$ (this pie is the $M$-orthogonal projection onto the range of $Q$). I would appreciate any hint on how to get from (1) to (2) (or vice versa).

I've tried to get from (2) to (1). First, $$(I-\tilde{\Pi})^TM(I-\tilde{\Pi})=M^{1/2}(I-\overline{\Pi})M^{1/2},$$ where now $$\overline{\Pi}=M^{1/2}Q(Q^TMQ)^{-1}Q^TM^{1/2}$$ is the orthogonal projection onto the range of $M^{1/2}Q$. So putting this to (2) and taking $w=M^{1/2}v$ gives $$ K=\max_w\frac{w^T(I-\overline{\Pi})w}{w^TM^{-1/2}(I-\Pi)M^{-1/2}w}, $$ which has already a sort of $M^{-1}$ in the denominator, but still not in the form of (1) and with a different projector in the numerator.

P.S.: I've asked a related question already, but the simply stated solution there does not much help me to get to (2) as the subspace over which the maximum is evaluated depends on $M$. What I'm actually looking for is a characterisation of (1) which does not change the subspace over which we maximise. In fact, (1) and (2) give that $$ K=\max_{w\in\mathcal{R}(I-\Pi)}\frac{w^Tw}{w^TM^{-1}w}=\max_{w\in\mathcal{R}(I-\Pi)}\frac{w^T(I-\tilde{\Pi})^TM(I-\tilde{\Pi})w}{w^Tw}. $$


After some playing with this problem, I think I've found a solution.

Let $U$ be an orthonormal basis of the range of $I-\Pi$, that is, $[Q,U]$ is a square orthogonal matrix such that $U^TQ=0$ and $I-\Pi=UU^T$. Then $$ \begin{split} K&=\max_v\frac{v^T(I-\Pi)v}{v^T(I-\Pi)M(I-\Pi)v}=\max_v\frac{v^TUU^Tv}{v^TUU^TM^{-1}UU^Tv} =\max_v\frac{v^T(U^TM^{-1}U)^{-1}v}{v^Tv}\\ &=\max_v\frac{v^T(U^TM^{-1}U)^{-1}U^TM^{-1}U(U^TM^{-1}U)^{-1}v}{v^Tv}\\ &=\max_v\frac{v^TU^TU(U^TM^{-1}U)^{-1}U^TM^{-1}MM^{-1}U(U^TM^{-1}U)^{-1}U^TUv}{v^TU^TUv}\\ &=\max_{v\in\mathcal{R}(I-\Pi)}\frac{v^T[U(U^TM^{-1}U)^{-1}U^TM^{-1}]M[M^{-1}U(U^TM^{-1}U)^{-1}U^T]v}{v^Tv}. \end{split} $$ What remains to show is that $$\tag{1}I-\tilde{\Pi}=I-Q(Q^TMQ)^{-1}Q^T\quad\text{and}\quad\Phi=M^{-1}U(U^TM^{-1}U)^{-1}U^T$$ are equal. It is easy to show that the square matrix $[U,MQ]$ is nonsingular. So $I-\Pi=\Phi$ if and only if $[U,MQ]^T(I-\Pi-\Phi)=0$. But this is easy to verify using $U^TQ=0$ and the expressions in (1).

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.