Let $X$ be a $\sigma$-subGaussian random vector on $\mathbb R^n$ (for large $n \ge 3$), meaning that the random variable $X^Tv$ is $\sigma$-subGaussian for every unit vector $v \in \mathbb R^n$. Consider the $n$-by-$n$ psd matrix $\Sigma := \mathbb E[UU^T]$, where $U := X/\|X\|_2$. It is clear that every eigenvalue of $\Sigma$ lies in the interval $[0, 1]$. In fact, $\text{tr}\Sigma \le 1$.
>**Question 1.** What is a good estimate for the largest eigenvalue of $\Sigma $ ?
>**Question 2.** Same question without the sub-Gaussianity assumption.
My Current approach
---
Let $\lambda$ be an eigenvalue of $\Sigma$ and $z$ be a unit vector in the corresponding eigenspace.
For any $\delta \in [0,1]$, let $G_\delta := \{x \in \mathbb R^n \mid |x^Tz| > \delta\}$. then
$$
\begin{split}
\lambda &= \lambda\|z\|^2 = z^T(\lambda z) = z^T\Sigma z = z^T E[UU^T]z = E[z^TUU^Tz] = E|U^Tz|^2\\
&= E[|U^Tz|^2 \mid U \in G_\delta]P(U \in G_\delta) + E[|U^Tz|^2 \mid U \in G^c_\delta)P(U \in G^c_\delta)\\
&\le P(U \in G_\delta) + \delta^2P(U \in G_\delta^c) = (1 - \delta^2)P(U \in G_\delta) + \delta^2.
\end{split}
$$
That is,
$$
\lambda \le (1-\delta^2)P(|U^Tz| > \delta) + \delta^2,\; \forall \delta \in [0, 1].
\tag{1}
$$
>Thus, if I had a bound on $P(|U^Tz| > \delta)$, I could plug it in (1) and then minimize over $\delta \in [0, 1]$ to get (a perhaps good) upper bound on $\lambda$.
For simplicity, suppose $X \sim \mathcal N(0,\sigma^2 I_n)$. Since $U$ is uniformly distributed on the unit $n$-sphere, it follows by symmetry that for every unit vector $z \in \mathbb R^d$, the random variable $U^Tz$ has the same distribution as $U_1$ (the first coordinate of the random vector $U$), which in turn (by the [Archimedean projection property][1]) has the same distribution as the first coordinate of a point draw uniformly in the unit ball in $\mathbb R^{n-1}$. Thus, $P(U_1 > \delta)$ is the probability that a random point in the unit ball in $\mathbb R^{n-2}$ lies in on given side of an equatorial hyperplane, we have
$$
\begin{split}
P(|U^Tz| > \delta) &= P(|U_1| > \delta)= 2P(U_1 > \delta) = 1-I\left(\delta;\frac{1}{2}, \frac{n-1}{2}\right)\\
&= I\left(1-\delta;\frac{n-1}{2},\frac{1}{2}\right),
\end{split}
\tag{2}
$$
$$
\begin{split}
P(|U^Tz| > \delta) &= 1-I\left(\delta;\frac{1}{2}, \frac{n-1}{2}\right) = I\left(1-\delta;\frac{n-1}{2},\frac{1}{2}\right),
\end{split}
\tag{2}
$$
where $I(t; a, b)$ is the normalized *incomplete beta function*, defined by $I_t(t; a, b) := B(t;a,b) / B(1; a, b)$, with $B(t; a, b):= \int_{0}^t s^{a-1}(1-s)^{b-1}ds$.
----
Edit: Bounding $P(|U^Tz| > \delta)$
===
>**Theorem ($U^Tz$ is sub-exponential! ).** *Let $U$ be uniformly distributed on the unit $n$-sphere and let $z$ be a fixed vector on this sphere. If $n$ is large enough, then for every $\delta \in [0, 1]$, it holds that*
$$
P(|U^Tz| > \delta) \le e^{-\frac{n-1}{4}\delta}.
\tag{3}
$$
*Consequently, we have the spectral bound*
$$
\lambda_{\max}(\Sigma) \le \min_{0 \le \delta \le 1}(1-\delta^2)e^{-\frac{n-1}{4}\delta} + \delta^2 \sim \frac{\log\log n}{n^2}.
\tag{*}
$$
*Proof.*
Let $p = I(1-\delta; 1/2, (n-1)/2)$.
It is known since [Temme (1992)][2] that for $p \in (0, 1)$ and **large** $a > 0$, the solution of the equation $p = I(t; a,b)$ is given (approximately) by
$$
t=t_p(a, b) \approx e^{-(1/a)Q_{1-p}(\Gamma(b,1))}, \tag{4}
$$
where $Q_{1-p}(\Gamma(b,1))$ is the $1-p$ quantile of the unit-scale gamma distribution with shape parameter $b$. Now by standard concentration results (e.g see Boucheron et al. textbook),
$$
Q_{1-p}(\Gamma(b,1)) \le \log(1/p) + \sqrt{2b\log(1/p)}. \tag{5}
$$
In particular, for $a=(n-1)/2$ and $b=1/2$ we get
$$
Q_{1-p}(\Gamma(1/2,1)) \le \log(1/p) + \sqrt{\log(1/p)} \le 2\log(1/p). \tag{6}
$$
Putting (2), (4), and (6) together and using the basic inequality $e^{-t} \ge 1-t\;\forall t > -1$, we see that
$$
\begin{split}
1-\delta &\ge t_{2p}\left((n-1)/2,1/2\right) \ge e^{-\frac{2Q_{1-2p}(\Gamma(1/2,1))}{n-1}} \ge e^{-\frac{2}{n-1}\left(\log\left(\frac{1}{2p}\right) + \sqrt{\log\left(\frac{1}{2p}\right)}\right)}\\
& \ge 1 - \frac{2\left(\log\left(\frac{1}{2p}\right) + \sqrt{\log\left(\frac{1}{2p}\right)}\right)}{n-1} \ge 1-\frac{4\log\left(\frac{1}{2p}\right)}{n-1},
\end{split}
$$
from which (3) follows upon combining with (2). Finally, (*) follows from (1) and (3) and the estimate obtained [here][3]. $\quad\quad\Box$
Below is a graphical visualization of the bound obtained with the above ingredients.
[![enter image description here][4]][4]
[1]: https://arxiv.org/abs/1504.02941
[2]: https://www.sciencedirect.com/science/article/pii/037704279290244R
[3]: https://math.stackexchange.com/a/3633732/168758
[4]: https://i.sstatic.net/LazHw.png