RMT for modified Wishard matrix $Y'Y$ (where $i$th row of $Y$ is zero if $|x_i^\top u| \le \theta$; else it equals $x_i$)

Question

Let $n$ and $d$ be positive integers tending to infinity such that $d/n \to \phi \in (0,\infty)$. Let $X$ be an $n \times d$ random matrix with iid rows $x_1,\ldots,x_n$ from $N(0, \Sigma)$, where $\Sigma$ satisfies usual spectral regularity conditions for RMT (bounded spectrum, etc.). Fix a unit-vector $u \in \mathbb R^d$ and a scalar $\theta\in [0,\infty]$, and form another $n \times d$ random matrix $Y$ with rows $y_1,\ldots,y_n$ as follows

$$ y_i = \begin{cases} 1,&\mbox{ if }|x_i^\top u| \le \theta,\\ 0,&\mbox{ else.} \end{cases} $$ Define $S_Y:=Y'Y/n$ and for any $\lambda > 0$, set $f(\lambda) := \operatorname{tr}(S_Y+\lambda I_d)^{-1}$.

Question. Is there a (system of) fixed-point equation(s) satisfied by $f(\lambda)$ ?

---

Observe that when $\theta=\infty$, we have $Y=X$, and we have the well-known fixed-point equation $$ f(\lambda) \simeq \operatorname{tr}(\Sigma+\kappa I_d)^{-1} = d/(1+\kappa), $$ where $\kappa$ satisfies the fixed-point equation $$ \kappa-\lambda = \phi \kappa / (1+\kappa), $$ which reminiscent of the well-known MP law.

Answer 1

Isotropic Case

The case $\Sigma=I_d$ is treated. For $z \sim N(0,I_d)$, let $D$ be the distribution of $z$ conditioned on $|z^\top u| \le \theta$. Then, the rows of $Y$ are iid from $D$. It is clear that $D = TN(-\theta,\theta) \oplus N(0,I_{d-1})$, where $TN(-\theta,\theta)$ is the truncated normal distribution on the interval $[-\theta,\theta]$. By universality of MP law, it follows that limiting spectral distribution (LSD) of $S:=Y^\top Y/n = (n'/n)Y^\top Y/n'$ is as if $Y$ were an $n \times n$ random matrix $\sigma_0 X_0$ with iid entries from $N(0,\sigma_0^2)$, where $\sigma_0^2 = n'/n$. This is MP, with variance parameter $\sigma^2$ and scale $$ \phi' = d/m = (d/n)(n/n') \simeq \phi / \sigma_0^2. $$

Let $M \subseteq [n]$ be the indices of the rows of $X$ which are actually selected. For any $z \in \mathbb C\setminus \mathbb R_+$. We have

$$ \begin{split} \mathbb E[\operatorname{tr}S(S - \lambda I)^{-1} \mid M] &\simeq \mathbb E\operatorname{tr} \sigma_0^2S_0(\sigma_0^2S_0 - z I)^{-1} =\mathbb E \operatorname{tr} S_0(S_0 - z' I)^{-1}\\ &\simeq \operatorname{tr}\Sigma(\Sigma - z'I)^{-1} = \frac{d}{1+\kappa'}, \end{split} $$ where $\Sigma = \sigma_0^2 I$, $\kappa' := \kappa(-z';\phi')$, $\lambda' := \lambda/\sigma_0^2$, and $$ \kappa(z;\phi) = \frac{z-(\phi-1) + \sqrt{(z-(\phi-1))^2 + 4z}}{2} $$

Likewise, we have $$ \begin{split} \mathbb E[\operatorname{tr}(S -z I)^{-1} \mid M] &\simeq \sigma_0^{-2}\mathbb E\operatorname{tr}(S_0 -z' I)^{-1} \simeq \sigma_0^{-2}\frac{\kappa'}{-z} \operatorname{tr}\Sigma(\Sigma + \kappa' I)^{-1} \\ &\simeq \frac{\kappa'}{-z}\operatorname{tr}(\Sigma + \kappa' I)^{-1} = -\frac{\kappa'}{z}\frac{d}{1+\kappa'}. \end{split} $$ We turn to the calculation of $\operatorname{tr} S^k(S-zI)^{-\ell}$ for any $k,\ell \in \mathbb Z$. Observe that \begin{align} \mathbb E[\operatorname{tr} S^k(S-z I)^{-\ell} \mid M] \simeq \sigma_0^{2(k-\ell)}\mathbb E\underbrace{\operatorname{tr} S_0^k(S_0-z' I)^{-\ell}}_{\text{use classical RMT here}}. \end{align}

It remains to take expectations over $n' \sim Bin(n,p)$, where $p = \mathbb P(|z^\top u| \le \theta) = 2\Phi(\theta)-1$, i.e

$$ \mathbb E\operatorname{tr} S^k(S-z I)^{-\ell} = \sum_{n'=0}^n {n \choose n'}p^{n'} (1-p)^{n-n'}\sigma_0^{2(k-\ell)}\mathbb E\operatorname{tr} S_0^k(S_0-z' I)^{-\ell}. $$

N.B. Curiously, it seems the value of the above sum is close to the value of the summand evaluated at $n' = np$ (maybe there is a statistical mechanical explanation for this ?).