$$ \mathbf{u} := \left( \mathbf{X}^H \mathbf{X} + \mathbf{I}_m + \mathbf{\lambda}\mathbf{D} \right)^{-1} \mathbf{X}^H \mathbf{y} $$


  • $\mathbf{X}$ is $n \times m$ semi-orthogonal matrix ($\mathbf{X} \mathbf{X}^H = \mathbf{I}_n$)

  • $\mathbf{D}$ is $m \times m$ diagonal matrix

  • $\mathbf{y}$ is $n \times 1$

  • $\mathbf{I}_m$ is $m \times m$ identity matrix

and $m \gg n$. Let

$$\rho(\lambda) := \mathbf{u}^H \mathbf{u}$$

Find $\lambda > 0$ such that $$\rho ' (\lambda) = 0$$

Appreciate any solutions or any numerical schemes for finding $\lambda$.


Working on the lines of the comments, I get expression for the derivative.

Let $G = X^HX+I+\lambda D$, assuming all entries of $D$ are real (as I am interested in it(I forgot to mention before)),

$$\frac{\partial \rho}{\partial \lambda} = -y^HXG^{-1}\Big(DG^{-1} + G^{-1}D\Big)G^{-1}X^Hy$$

  • 1
    $\begingroup$ In general $\frac{\partial A^{-1}}{\partial \lambda}=A^{-1}\frac{\partial A}{\partial\lambda}A^{-1}$. With this it should be possible to get an expression for $\frac{\partial \rho}{\partial \lambda}$. $\endgroup$ – user100927 Apr 26 at 9:32
  • $\begingroup$ @user100927 : Thanks. I have done that. I don't know how to solve from there, the matrix equation for the scalar $\lambda$. $\endgroup$ – Rajesh Dachiraju Apr 26 at 9:48
  • $\begingroup$ @RodrigodeAzevedo : have added that. $\endgroup$ – Rajesh Dachiraju Apr 29 at 8:59
  • $\begingroup$ How large are $m$ and $n$? Why not use symbolic computation? Brute, but it may work. $\endgroup$ – Rodrigo de Azevedo Apr 30 at 8:48
  • $\begingroup$ @RodrigodeAzevedo : I am in search of an algorithm to compute $\lambda$, given the matrices. The sizes $n$ and $m$ could be arbitrary, only condition being $m > n$. Also generally $m >> n$, if that helps to reduce algorithm complexity. $\endgroup$ – Rajesh Dachiraju Apr 30 at 9:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.