I am interested in covaraince matrix estimation. In brief: I have two estimates of the covariance matrix, and now I want to form a bona fide convex combination of the two.

Background: I have studied the work on regularization of covariance matrices. Assume the $R$ is the true $p\times p$ covariance matrix, and that I have $N$ observed vectors with covariance $R$. Then, a common technique, deeply studied by Ledoit and Wolf, is to regularize the estimate according to $$(1-\rho)\hat{S} + \rho I$$ where $\hat{S}$ is the sample covariance matrix. The optimal $\rho$, in a quadratic mean sense, depends on $R$ which is unknown. Therefore, a bona fide estimator is derived which is asymptotically consistent, that is as $N$ and $p$ grows, the bona fide estimator is as good as the optimal which requires knowledge of $R$.

Background of my problem: I have two sets of data, one set where N is huge but where the reliability is fairly low, and another set where N is smaller, but the accuracy per observed vector is higher than for the first set (this may seem as an utterly confusing statement, but the reason is that that in the first set, I observe $Hx+Gy$ where both $x$ and $y$ are random, and $R=HH^{H}+GG^{H}$. In the second set, I observe $Ht+Gy$, where $t$ are known training vectors so that the randomness is less.) This leaves me with two sample covariance matrices, $\hat{S}_1$ and $\hat{S}_2$, and my next step is to form a convex combination $$\hat{S} = (1-\rho)\hat{S}_1+\rho \hat{S}_2$$ I can do this, but I do not get a bona fide estimator.

Question: In the regularization framework of, e.g., LW, is it important that one shrink towards the identity? Or will the general framework hold true for a convex combination of two arbitrary matrices?

  • $\begingroup$ Can you give a reference to the wprk by Ledoit and Wolf that you cite? $\endgroup$ Nov 23 '12 at 15:56
  • $\begingroup$ In what sense is $\hat{S}$ not a "bona fide" estimator? $\endgroup$
    – R Hahn
    Nov 23 '12 at 21:17

Klart jag kan :-) Sure, see for example "Honey, I shrank the covariance matrix" by Ledoit and wolf. Another, more engineer oriented paper is Chen, XXX, Eldar, and Hero III "MMSE shrinkage of covariance matrices"

  • $\begingroup$ Since the optimal $\rho$ is dependent on $R$. I need something that is independent in online applications. $\endgroup$ Nov 24 '12 at 12:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.