Let $Z=[z_1, \dots z_n]$ be a $d \times n$ matrix, where the $z_i$'s are iid random vactors with mean $\mu \in \mathbb{R}^d$ and $d \times d$ (population) covariance matrix $\Sigma$, but the entries $z_{ij}'s $ are not necessarily iid. Consider the (unscaled) sample covariance matrix $C:= ZZ' \in \mathbb{R}^{d \times d}$. I was wondering whether we have any results on the upper and lower bounds of $\lambda_1(C), \lambda_d{(C)}$ as a function of $n$ and $d$. I'm not assuming anything like the asyptotic regime in random matrix theory, so for example, no assumption like $n, d \to \infty, d/n \to c$. I'd rather have a non-asymtotic result, showing the dependence on $d, n$. Thank you and references would be really helpful!
$\begingroup$
$\endgroup$
asked Mar 17, 2020 at 21:57
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
6
If the random vectors are isotropic (meaning $E[z_i z_i^T]=I$), you can use the lower bound derived by Pavel Yaskov (2014):
Abstract: We provide tight lower bounds on the smallest eigenvalue of a sample covariance matrix of a centred isotropic random vector under weak or no assumptions on its components.
answered Mar 17, 2020 at 22:26
-
$\begingroup$ Thanks, this seems to be interesting for tight lower bounds. Vershynin's book "High Dimensional Probability with Application to data Science" gives some upper and lower bounds in the order of $\sqrt n +/- C \sqrt d$ (Theorem 4.6.1, P.98) , but assumes itotropic again. But I'll check this paper too... I'm just not sure how realistic this isotropic assumtion on $z_i's$ are, for real data the "features" $z_{ij}, z_{ik}$ are going to be correlated. $\endgroup$ Commented Mar 17, 2020 at 22:39
-
1$\begingroup$ isotropy allows for correlations, doesn't it? it just means that that the vector has no preferential direction. $\endgroup$ Commented Mar 18, 2020 at 7:11
-
$\begingroup$ okay, let me try to understand this. Say in my sample is $\{z_1,...z_n\}, z_i's$, are iid. Now if I assume that: each $z_i$ isotropic, then the polpulation covariance $cov(z_i) = \sigma^2 I_d$ (right?). This means (1) $cov(z_{ij}, z_{ik}) = 0 \forall j \ne k$. This means the $d$ features/covariates for each $z_i$ are pairwise uncorrelated (but not pairwise independent, except if they're normal), and also that (2) $Var(z_{ij}) = \sigma^2 \forall j ,$ so the all the individual features for each $z_i$ have same variance. Isn't it unrealistic to have uncorrelated features with same variances? $\endgroup$ Commented Mar 18, 2020 at 11:31
-
1$\begingroup$ the isotropy assumption has been relaxed by Yaskov in sciencedirect.com/science/article/abs/pii/S0024379516300714 $\endgroup$ Commented Mar 18, 2020 at 13:13
-
$\begingroup$ Thanks for the link -it's useful! $\endgroup$ Commented Mar 18, 2020 at 14:42