MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am interested in covaraince matrix estimation. In brief: I have two estimates of the covariance matrix, and now I want to form a bona fide convex combination of the two.

Background: I have studied the work on regularization of covariance matrices. Assume the $R$ is the true $p\times p$ covariance matrix, and that I have $N$ observed vectors with covariance $R$. Then, a common technique, deeply studied by Ledoit and Wolf, is to regularize the estimate according to $$(1-\rho)\hat{S} + \rho I$$ where $\hat{S}$ is the sample covariance matrix. The optimal $\rho$, in a quadratic mean sense, depends on $R$ which is unknown. Therefore, a bona fide estimator is derived which is asymptotically consistent, that is as $N$ and $p$ grows, the bona fide estimator is as good as the optimal which requires knowledge of $R$.

Background of my problem: I have two sets of data, one set where N is huge but where the reliability is fairly low, and another set where N is smaller, but the accuracy per observed vector is higher than for the first set (this may seem as an utterly confusing statement, but the reason is that that in the first set, I observe $Hx+Gy$ where both $x$ and $y$ are random, and $R=HH^{H}+GG^{H}$. In the second set, I observe $Ht+Gy$, where $t$ are known training vectors so that the randomness is less.) This leaves me with two sample covariance matrices, $\hat{S}_1$ and $\hat{S}_2$, and my next step is to form a convex combination $$\hat{S} = (1-\rho)\hat{S}_1+\rho \hat{S}_2$$ I can do this, but I do not get a bona fide estimator.

Question: In the regularization framework of, e.g., LW, is it important that one shrink towards the identity? Or will the general framework hold true for a convex combination of two arbitrary matrices?

share|cite|improve this question
Can you give a reference to the wprk by Ledoit and Wolf that you cite? – kjetil b halvorsen Nov 23 '12 at 15:56
In what sense is $\hat{S}$ not a "bona fide" estimator? – R Hahn Nov 23 '12 at 21:17

Klart jag kan :-) Sure, see for example "Honey, I shrank the covariance matrix" by Ledoit and wolf. Another, more engineer oriented paper is Chen, XXX, Eldar, and Hero III "MMSE shrinkage of covariance matrices"

share|cite|improve this answer
Since the optimal $\rho$ is dependent on $R$. I need something that is independent in online applications. – pierre robert Nov 24 '12 at 12:08

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.