1
$\begingroup$

Consider the generalized eigenvalue problem :

$ \Sigma_{XY} \Sigma_{YX} {W} = \lambda \Sigma_{XX} {W} $

where $\Sigma_{XX} $ and $\Sigma_{XY}$ are sample covariance matrices are of the matrices $X$ and $Y$ $\in M_{n}(\mathbb{R})$

When matrices $\Sigma_{XX}$ and/or $\Sigma_{XY}$ respectively are ill-conditionned, their inverses became unreliable. To deal with this problem, a regularization step is included. Specifically, one solves the following generalized eigenvalue problem:

$\Sigma_{XY} \Sigma_{YX} {W} = \lambda (\Sigma_{XX}+\alpha_{X} {I}) {W} $

with $ \alpha_{X}>0 $.

Is there a method to determine an optimal value of $ \alpha_{X}$ such that the residual error is the lowest ?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Is this a problem with large and sparse matrices? Because otherwise you shouldn't be doing inverses to compute eigenvalues, but rather use a backward stable method such as QZ, which avoids the issue completely. $\endgroup$
    – Federico Poloni
    Commented Feb 12, 2016 at 21:13
  • $\begingroup$ It is a problem with large matrices but they are not sparse. And thank you for QZ, i did not know this method. You mean that with QZ method, one can solved this generalized eigenvalue problem whithout adding the regularization parameter ? $\endgroup$
    – user41037
    Commented Feb 12, 2016 at 21:55
  • $\begingroup$ The issue is also that both matrices $\Sigma_{XX}$ and $\Sigma_{XY}\Sigma_{YX}$ are ill-conditioned. $\endgroup$
    – user41037
    Commented Feb 12, 2016 at 23:11
  • 1
    $\begingroup$ Yes, QZ uses no inverses, it is backward stable, and it is implemented in most scientific computing software (Matlab, R, scipy...). The only problem with it is that it is not backward stable in a way that preserves symmetry, so the computed eigenvalues may have a small imaginary part instead of being real (but I think that if you have formed the product $\Sigma_{XX}$ you have already lost hope of preserving realness). Maybe you can fix this, too, with a generalized SVD, if I understand your notation correctly. Check Section 8.7 of Golub-Van Loan, Matrix computations, 4th ed. $\endgroup$
    – Federico Poloni
    Commented Feb 13, 2016 at 8:21

0

Reset to default

You must log in to answer this question.