2
$\begingroup$

Let $\Omega$ be a symmetric and positive definite matrix. From a test of hypothesis I know that some eigenvalues are likely to be equal (the test also suggests which eigenvalues). Do you have any suggestions for obtaining the matrix, say $\tilde{\Omega}$, that can be obtained by fixing those eigenvalues to be equal? In other words, I would expect a matrix, quite "close" to $\Omega$, but not exactly the same.

Any suggestions is really appreciated.

Improve this question
$\endgroup$
8
  • $\begingroup$ So you are looking for the closest matrix to a given one with repeated eigenvalues? How many? In which metric? $\endgroup$
    – Federico Poloni
    Commented Feb 4, 2021 at 21:47
  • $\begingroup$ Hi Federico, it is exactly the point. Actually, a priori I don't know how many eigenvalues are equal. I am interested in a very general strategy for obtaining such $\tilde{\Omega}$, any metric could be justifiable. $\endgroup$
    – meie73
    Commented Feb 5, 2021 at 9:02
  • $\begingroup$ This sounds more like a modelling problem. What do you know, apart from the matrix entries? Do you have a 'typical' level of noise, for instance? What would you do if the eigenvalues are logarithmically distributed, e.g., $diag(1, 1.1, 1.11, 1.111, 1.11111, ...)$? $\endgroup$
    – Federico Poloni
    Commented Feb 5, 2021 at 9:08
  • $\begingroup$ I don't know anything more that the matrix $\Omega$. Essentially, it is an estimated covariance matrix of a multivariate model. But if some eigenvalues are equal, I might have problems in terms of identification of certain parameters of the model bla bla bla. Let a statistical test says $k$ eigenvalues, that look like very similar, are effectively equivalent (from a statistical point of view). I would like to see the consequences on my identification problem when the eigenvalues are exactly the same. For this reason, I would like to work with the "closest" matrix to the original $\Omega$. $\endgroup$
    – meie73
    Commented Feb 5, 2021 at 9:22
  • $\begingroup$ But then your test tells you $k$, at least? So the problem is, given $k$ and $\Omega$, find the closest matrix to $\Omega$ (in some metric) with $k$ repeated eigenvalues? $\endgroup$
    – Federico Poloni
    Commented Feb 5, 2021 at 9:26

2 Answers 2

Reset to default
1
$\begingroup$

The average (centroid) $\lambda = \frac{\lambda_1 + \dots + \lambda_k}{k}$ minimizes the sum of squared differences $f(\lambda) = \sum_{i=1}^k (\lambda - \lambda_i)^2$.

This suggests an algorithm, which is essentially what Louis Deaett suggests, if I understand correctly:

  • compute the diagonalization $\Omega = V\Lambda V^{-1}$.
  • permute eigenvalues and eigenvectors so that the sum of squared distances from the centroid $f(\lambda)$ is minimized by the first $k$ eigenvalues. It is not clear how to do this step in optimal time, even if the solution may be evident in the 'eyeball norm' in many practical cases.
  • replace those $k$ eigenvalues with their mean, to obtain $\tilde{\Omega} = V\tilde{\Lambda}V^{-1}$.

I conjecture that this algorithm gives you the optimal answer in the Frobenius norm $\|\Omega - \tilde{\Omega}\|_F^2 = \sum_{i,j} (\Omega_{ij} - \tilde{\Omega}_{ij})^2$; based on experience on similar problems, it looks like Weyl's inequalities for eigenvalues could be used to give a proof.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Dear Federico, thanks a lot for your suggestion. I will try to provide a proof for this intuitive result. $\endgroup$
    – meie73
    Commented Feb 9, 2021 at 8:02
0
$\begingroup$

I'm not sure if this is what you're after, but your positive definite (symmetric) matrix is going to be diagonalized by some unitary matrix $U$. So $U^*\Omega U=\Lambda$ for some diagonal matrix $\Lambda$ of its eigenvalues. This gives $U\Lambda U^*= \Omega$.

It sounds like perhaps what you are after is $\tilde\Omega=U\tilde\Lambda U^*$, where the difference between $\Lambda$ and $\tilde\Lambda$ is that in $\tilde\Lambda$, the two eigenvalues that were ``close'' have been replaced by two values that are equal.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks Louis for your reply. This was my first strategy to attack the problem. For sure, it is very simple and fast, but it is based on fixing the eigenvalues that are potentially equal to a value that I don't know. If from the statistical test I get the result that $k$ eigenvalues are equal, what shoud I do, fixing them to the average of these $k$ eigenvalues? Maybe, an iterative procedure could help in soving the problem, but I was wondering whether there was some analytical result in matrix algebra. Or maybe, some specific algorithm already treated in the literature. $\endgroup$
    – meie73
    Commented Feb 5, 2021 at 9:11
  • $\begingroup$ To your question, yes I think that is the thing left to determine, and it may depend on your application/context. The gist of Federico Poloni's answer is to choose this value so as to minimize the norm difference of your $\Omega$ and your $\tilde\Omega$, which is a natural idea. If you have more information from your application, then maybe something different makes sense. For example, maybe the application suggests that the two eigenvalues should not be given equal weight? I'm speculating, but there probably isn't more to say in full generality than what Federico has pointed out. $\endgroup$
    – Louis Deaett
    Commented Feb 5, 2021 at 18:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .