Intuitive understanding of the Stieltjes transform - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-24T13:37:22Z http://mathoverflow.net/feeds/question/79109 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/79109/intuitive-understanding-of-the-stieltjes-transform Intuitive understanding of the Stieltjes transform 2011-10-25T19:26:25Z 2011-10-25T23:59:23Z <p>I have been using random matrix theory in signal processing and have some trouble <em>understanding</em> what the Stieltjes transform does. </p> <p>The gist of my work is that I have an $N\times N$ true covariance matrix of the population with $k$ eigenvalues $\lambda_i>1,\ i=\lbrace 1,\ldots,k\rbrace$, corresponding to the signals and the remaining $N-k$ eigenvalues are all $1$, corresponding to the background noise. Assuming Gaussian statistics, I'm deriving the eigenvalue density of the sample covariance matrix of a set of observations from this population, which then is used to detect the signals.</p> <p>From <a href="http://www4.ncsu.edu/~jack/strong.pdf" rel="nofollow">J. W. Silverstein, <em>“Strong convergence of the empirical dis- tribution of eigenvalues of large dimensional random ma- trices”</em>, J. Multivar. Anal. <strong>54</strong>, 175–192 (1995)</a>, I understand that the Stieltes transform of the density of the sample covariance matrix can be written as </p> <p>$$m=\sum_{i=1}^{N\to\infty} \frac{1}{\lambda_i (1-c-c z m)-z}$$</p> <p>where $c$ is the ratio of number of dimensions $N$ to number of observations $M$ as $N,M\to\infty$. Taking it as a polynomial in $m$ and looking for solutions in the imaginary plane, I can retrieve my density using the inverse transform. </p> <p>So far, all this is well and good and I have my numerical routines that solve this for any model I input and give me the correct density. However, my question is more fundamental, as to the role that Stieltjes transforms plays here. How does one go from the <a href="http://en.wikipedia.org/wiki/Stieltjes_transformation" rel="nofollow">definition in wikipedia</a></p> <p>$$S_\rho(z)=\int_I \frac{\rho(x)}{z-x}\ dx$$</p> <p>where $I$ is the interval of support of $\rho(x)$ to the equation in Silverstein's paper. I understand that Stieltes transforms deal with measures of density and if I had in fact taken the density of the population covariance matrix, i.e., $\rho(\lambda)=\sum_{i=1}^N \delta(\lambda-\lambda_i)$, its Stieltjes transform would've looked like</p> <p>$$S=\frac{1}{N}\sum_{i=1}^N\frac{1}{z-\lambda_i}$$</p> <p>So I can see the similarity with the first equation above (barring a sign change that's trivial to take care of), <strong>but don't understand how the $(1-c-czm)$ term comes into the picture and how that helps map the density of the eigenvalues of the population covariance matrix to those of the sample.</strong></p> <p>I've tried reading <a href="http://terrytao.wordpress.com/2010/02/02/254a-notes-4-the-semi-circular-law/" rel="nofollow">Terry Tao's blog</a> on the topic, and while that was enlightening, it didn't fix this conceptual gap. It seemed like the original 1965 paper by Marchenko and Pastur dealt with these from a more first principles approach than Silverstein. However, that paper is too dense to tackle, and I'm not sure how using partial differential equations like they do help solve it. Perhaps I'm missing something here because I'm not a mathematician, but I'd appreciate someone helping me fill this gap.</p> http://mathoverflow.net/questions/79109/intuitive-understanding-of-the-stieltjes-transform/79129#79129 Answer by Terry Tao for Intuitive understanding of the Stieltjes transform Terry Tao 2011-10-25T22:03:27Z 2011-10-25T22:03:27Z <p>Firstly, the equation you attribute to Silverstein (and is sometimes known as the "self-consistent equation" for the Stieltjes transform) is not exact, but only asymptotically valid in the limit $n \to \infty$. The definition given in Wikipedia is the exact formula. (Your final formula, by the way, is missing a normalisation factor of either $1/n$ or $1/p$, depending on conventions.)</p> <p>The self-consistent equation arises by viewing the Stieltjes transform as a trace</p> <p>$$S = \frac{1}{n} \hbox{tr}( A - zI )^{-1} = \frac{1}{n} \sum_{j=1}^n ((A-zI)^{-1})_{jj}.$$</p> <p>It is possible to solve for the $jj^{th}$ component $((A-zI)^{-1})_{jj}$ of the resolvent $(A-zI)$ using the method of <a href="http://en.wikipedia.org/wiki/Schur_complement" rel="nofollow">Schur complements</a>, in terms of an expression involving the inverse of an $n-1 \times n-1$ minor of $A$, which in turn can be approximated in terms of the eigenvalues of that minor. The eigenvalues of the minor can in turn be estimated by the eigenvalues of the original matrix by means of the <a href="http://en.wikipedia.org/wiki/Min-max_theorem#Cauchy_interlacing_theorem" rel="nofollow">Cauchy interlacing law</a>, and the resulting expression involving these eigenvalues turns out to be a function of the original Stieltjes transform, leading to the (asymptotic) formula mentioned in Silverstein's paper (and also in my blog post you mentioned, in the case of Wigner matrices).</p> <p>The Stieltjes transform can be viewed as a complexification of the spectral measure. Indeed, if one looks at the "jump" in the Stieltjes transform as one passes from the upper half plane to the lower half plane, this jump is (up to some factors of $\pi$) essentially the spectral measure. The real part of the Stieltjes transform is thus essentially the harmonic extension of spectral measure (or, if you like, a smoothed out version of the eigenvalue counting function $N_I$), and the imaginary part is its harmonic conjugate. As such, it neatly packages the spectral information in a way that can be easily manipulated by the methods of complex analysis.</p> <p>Another way to view the Stieltjes transform is as a sort of moment generating function for the spectral density. This is basically the viewpoint taken in free probability (see my <a href="http://terrytao.wordpress.com/2010/02/10/245a-notes-5-free-probability/" rel="nofollow">later lecture notes</a> on this topic).</p>