Let $Z=[z_1, \dots z_n]$ be a $d \times n$ matrix, where the $z_i$'s are iid random vactors with mean $\mu \in \mathbb{R}^d$ and $d \times d$ (population) covariance matrix $\Sigma$, but the entries $z_{ij}'s $ are not necessarily iid. Consider the (unscaled) sample covariance matrix $C:= ZZ' \in \mathbb{R}^{d \times d}$. I was wondering whether we have any results on the upper and lower bounds of $\lambda_1(C), \lambda_d{(C)}$ as a function of $n$ and $d$. I'm not assuming anything like the asyptotic regime in random matrix theory, so for example, no assumption like $n, d \to \infty, d/n \to c$. I'd rather have a non-asymtotic result, showing the dependence on $d, n$. Thank you and references would be really helpful!


If the random vectors are isotropic (meaning $E[z_i z_i^T]=I$), you can use the lower bound derived by Pavel Yaskov (2014):

Abstract: We provide tight lower bounds on the smallest eigenvalue of a sample covariance matrix of a centred isotropic random vector under weak or no assumptions on its components.

  • $\begingroup$ Thanks, this seems to be interesting for tight lower bounds. Vershynin's book "High Dimensional Probability with Application to data Science" gives some upper and lower bounds in the order of $\sqrt n +/- C \sqrt d$ (Theorem 4.6.1, P.98) , but assumes itotropic again. But I'll check this paper too... I'm just not sure how realistic this isotropic assumtion on $z_i's$ are, for real data the "features" $z_{ij}, z_{ik}$ are going to be correlated. $\endgroup$ Mar 17 '20 at 22:39
  • 1
    $\begingroup$ isotropy allows for correlations, doesn't it? it just means that that the vector has no preferential direction. $\endgroup$ Mar 18 '20 at 7:11
  • $\begingroup$ okay, let me try to understand this. Say in my sample is $\{z_1,...z_n\}, z_i's$, are iid. Now if I assume that: each $z_i$ isotropic, then the polpulation covariance $cov(z_i) = \sigma^2 I_d$ (right?). This means (1) $cov(z_{ij}, z_{ik}) = 0 \forall j \ne k$. This means the $d$ features/covariates for each $z_i$ are pairwise uncorrelated (but not pairwise independent, except if they're normal), and also that (2) $Var(z_{ij}) = \sigma^2 \forall j ,$ so the all the individual features for each $z_i$ have same variance. Isn't it unrealistic to have uncorrelated features with same variances? $\endgroup$ Mar 18 '20 at 11:31
  • 1
    $\begingroup$ the isotropy assumption has been relaxed by Yaskov in sciencedirect.com/science/article/abs/pii/S0024379516300714 $\endgroup$ Mar 18 '20 at 13:13
  • $\begingroup$ Thanks for the link -it's useful! $\endgroup$ Mar 18 '20 at 14:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.