Skip to main content
fixed small typos and missing factors
Source Link
user avatar
user avatar

I have been using random matrix theory in signal processing and have some trouble understanding what the Stieltjes transform does.

The gist of my work is that I have an $N\times N$ true covariance matrix of the population with $k$ eigenvalues $\lambda_i>1,\ i=\lbrace 1,\ldots,k\rbrace$, corresponding to the signals and the remaining $N-k$ eigenvalues are all $1$, corresponding to the background noise. Assuming Gaussian statistics, I'm deriving the eigenvalue density of the sample covariance matrix of a set of observations from this population, which then is used to detect the signals.

From J. W. Silverstein, “Strong convergence of the empirical dis- tribution of eigenvalues of large dimensional random ma- trices”, J. Multivar. Anal. 54, 175–192 (1995), I understand that the Stieltes transform of the density of the sample covariance matrix can be written as

$$m=\sum_{i=1}^N \frac{1}{\lambda_i (1-c-c z m)-z}$$$$m=\sum_{i=1}^{N\to\infty} \frac{1}{\lambda_i (1-c-c z m)-z}$$

where $c$ is the ratio of number of dimensions $N$ to number of observations $M$ as $N,M\to\infty$. Taking it as a polynomial in $m$ and looking for solutions in the imaginary plane, I can retrieve my density using the inverse transform.

So far, all this is well and good and I have my numerical routines that solve this for any model I input and give me the correct density. However, my question is more fundamental, as to the role that Stieltjes transforms plays here. How does one go from the definition in wikipedia

$$S_\rho(z)=\int_I \frac{\rho(x)}{z-x}\ dx$$

where $I$ is the interval of support of $\rho(x)$ to the equation in Silverstein's paper?. I understand that Stieltes transforms deal with measures of density and if I had in fact taken the density of the population covariance matrix, i.e., $\rho(\lambda)=\sum_{i=1}^N \delta(\lambda-\lambda_i)$, its Stieltjes transform would've looked like

$$S=\sum_{i=1}^N\frac{1}{z-\lambda_i}$$$$S=\frac{1}{N}\sum_{i=1}^N\frac{1}{z-\lambda_i}$$

So I can see the similarity with the first equation above (barring a sign change that's trivial to take care of), but don't understand how the $(1-c-czm)$ term comes into the picture and how that helps map the density of the eigenvalues of the population covariance matrix to those of the sample.

I've tried reading Terry Tao's blog on the topic, and while that was enlightening, it didn't fix this conceptual gap. It seemed like the original 1965 paper by Marchenko and Pastur dealt with these from a more first principles approach than Silverstein. However, that paper is too dense to tackle, and I'm not sure how using partial differential equations like they do help solve it. Perhaps I'm missing something here because I'm not a mathematician, but I'd appreciate someone helping me fill this gap.

I have been using random matrix theory in signal processing and have some trouble understanding what the Stieltjes transform does.

The gist of my work is that I have an $N\times N$ true covariance matrix of the population with $k$ eigenvalues $\lambda_i>1,\ i=\lbrace 1,\ldots,k\rbrace$, corresponding to the signals and the remaining $N-k$ eigenvalues are all $1$, corresponding to the background noise. Assuming Gaussian statistics, I'm deriving the eigenvalue density of the sample covariance matrix of a set of observations from this population, which then is used to detect the signals.

From J. W. Silverstein, “Strong convergence of the empirical dis- tribution of eigenvalues of large dimensional random ma- trices”, J. Multivar. Anal. 54, 175–192 (1995), I understand that the Stieltes transform of the density of the sample covariance matrix can be written as

$$m=\sum_{i=1}^N \frac{1}{\lambda_i (1-c-c z m)-z}$$

where $c$ is the ratio of number of dimensions $N$ to number of observations $M$ as $N,M\to\infty$. Taking it as a polynomial in $m$ and looking for solutions in the imaginary plane, I can retrieve my density using the inverse transform.

So far, all this is well and good and I have my numerical routines that solve this for any model I input and give me the correct density. However, my question is more fundamental, as to the role that Stieltjes transforms plays here. How does one go from the definition in wikipedia

$$S_\rho(z)=\int_I \frac{\rho(x)}{z-x}\ dx$$

where $I$ is the interval of support of $\rho(x)$ to the equation in Silverstein's paper? I understand that Stieltes transforms deal with measures of density and if I had in fact taken the density of the population covariance matrix, i.e., $\rho(\lambda)=\sum_{i=1}^N \delta(\lambda-\lambda_i)$, its Stieltjes transform would've looked like

$$S=\sum_{i=1}^N\frac{1}{z-\lambda_i}$$

So I can see the similarity with the first equation above (barring a sign change that's trivial to take care of), but don't understand how the $(1-c-czm)$ term comes into the picture and how that helps map the density of the eigenvalues of the population covariance matrix to those of the sample.

I've tried reading Terry Tao's blog on the topic, and while that was enlightening, it didn't fix this conceptual gap. It seemed like the original 1965 paper by Marchenko and Pastur dealt with these from a more first principles approach than Silverstein. However, that paper is too dense to tackle, and I'm not sure how using partial differential equations like they do help solve it. Perhaps I'm missing something here because I'm not a mathematician, but I'd appreciate someone helping me fill this gap.

I have been using random matrix theory in signal processing and have some trouble understanding what the Stieltjes transform does.

The gist of my work is that I have an $N\times N$ true covariance matrix of the population with $k$ eigenvalues $\lambda_i>1,\ i=\lbrace 1,\ldots,k\rbrace$, corresponding to the signals and the remaining $N-k$ eigenvalues are all $1$, corresponding to the background noise. Assuming Gaussian statistics, I'm deriving the eigenvalue density of the sample covariance matrix of a set of observations from this population, which then is used to detect the signals.

From J. W. Silverstein, “Strong convergence of the empirical dis- tribution of eigenvalues of large dimensional random ma- trices”, J. Multivar. Anal. 54, 175–192 (1995), I understand that the Stieltes transform of the density of the sample covariance matrix can be written as

$$m=\sum_{i=1}^{N\to\infty} \frac{1}{\lambda_i (1-c-c z m)-z}$$

where $c$ is the ratio of number of dimensions $N$ to number of observations $M$ as $N,M\to\infty$. Taking it as a polynomial in $m$ and looking for solutions in the imaginary plane, I can retrieve my density using the inverse transform.

So far, all this is well and good and I have my numerical routines that solve this for any model I input and give me the correct density. However, my question is more fundamental, as to the role that Stieltjes transforms plays here. How does one go from the definition in wikipedia

$$S_\rho(z)=\int_I \frac{\rho(x)}{z-x}\ dx$$

where $I$ is the interval of support of $\rho(x)$ to the equation in Silverstein's paper. I understand that Stieltes transforms deal with measures of density and if I had in fact taken the density of the population covariance matrix, i.e., $\rho(\lambda)=\sum_{i=1}^N \delta(\lambda-\lambda_i)$, its Stieltjes transform would've looked like

$$S=\frac{1}{N}\sum_{i=1}^N\frac{1}{z-\lambda_i}$$

So I can see the similarity with the first equation above (barring a sign change that's trivial to take care of), but don't understand how the $(1-c-czm)$ term comes into the picture and how that helps map the density of the eigenvalues of the population covariance matrix to those of the sample.

I've tried reading Terry Tao's blog on the topic, and while that was enlightening, it didn't fix this conceptual gap. It seemed like the original 1965 paper by Marchenko and Pastur dealt with these from a more first principles approach than Silverstein. However, that paper is too dense to tackle, and I'm not sure how using partial differential equations like they do help solve it. Perhaps I'm missing something here because I'm not a mathematician, but I'd appreciate someone helping me fill this gap.

Source Link
user avatar
user avatar

Intuitive understanding of the Stieltjes transform

I have been using random matrix theory in signal processing and have some trouble understanding what the Stieltjes transform does.

The gist of my work is that I have an $N\times N$ true covariance matrix of the population with $k$ eigenvalues $\lambda_i>1,\ i=\lbrace 1,\ldots,k\rbrace$, corresponding to the signals and the remaining $N-k$ eigenvalues are all $1$, corresponding to the background noise. Assuming Gaussian statistics, I'm deriving the eigenvalue density of the sample covariance matrix of a set of observations from this population, which then is used to detect the signals.

From J. W. Silverstein, “Strong convergence of the empirical dis- tribution of eigenvalues of large dimensional random ma- trices”, J. Multivar. Anal. 54, 175–192 (1995), I understand that the Stieltes transform of the density of the sample covariance matrix can be written as

$$m=\sum_{i=1}^N \frac{1}{\lambda_i (1-c-c z m)-z}$$

where $c$ is the ratio of number of dimensions $N$ to number of observations $M$ as $N,M\to\infty$. Taking it as a polynomial in $m$ and looking for solutions in the imaginary plane, I can retrieve my density using the inverse transform.

So far, all this is well and good and I have my numerical routines that solve this for any model I input and give me the correct density. However, my question is more fundamental, as to the role that Stieltjes transforms plays here. How does one go from the definition in wikipedia

$$S_\rho(z)=\int_I \frac{\rho(x)}{z-x}\ dx$$

where $I$ is the interval of support of $\rho(x)$ to the equation in Silverstein's paper? I understand that Stieltes transforms deal with measures of density and if I had in fact taken the density of the population covariance matrix, i.e., $\rho(\lambda)=\sum_{i=1}^N \delta(\lambda-\lambda_i)$, its Stieltjes transform would've looked like

$$S=\sum_{i=1}^N\frac{1}{z-\lambda_i}$$

So I can see the similarity with the first equation above (barring a sign change that's trivial to take care of), but don't understand how the $(1-c-czm)$ term comes into the picture and how that helps map the density of the eigenvalues of the population covariance matrix to those of the sample.

I've tried reading Terry Tao's blog on the topic, and while that was enlightening, it didn't fix this conceptual gap. It seemed like the original 1965 paper by Marchenko and Pastur dealt with these from a more first principles approach than Silverstein. However, that paper is too dense to tackle, and I'm not sure how using partial differential equations like they do help solve it. Perhaps I'm missing something here because I'm not a mathematician, but I'd appreciate someone helping me fill this gap.