Statistics for Haar measure of random matrices? - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-21T23:08:20Z http://mathoverflow.net/feeds/question/77332 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/77332/statistics-for-haar-measure-of-random-matrices Statistics for Haar measure of random matrices? Jiahao Chen 2011-10-06T05:37:23Z 2011-10-08T05:40:04Z <p>Let's say I have $M$ samples of $N\times N$ real orthogonal matrices. What statistics can I calculate to test if they could have been drawn from a distribution consistent with Haar measure over $O(N)$?</p> <p>This question is related to <a href="http://mathoverflow.net/questions/76295/intuition-for-haar-measure-of-random-matrix" rel="nofollow">this previous question</a>.</p> http://mathoverflow.net/questions/77332/statistics-for-haar-measure-of-random-matrices/77362#77362 Answer by Mark Meckes for Statistics for Haar measure of random matrices? Mark Meckes 2011-10-06T13:42:20Z 2011-10-06T18:06:41Z <p>I'm guessing that you meant to write "real orthogonal matrices" without "symmetric", since the set of symmetric orthogonal matrices has Haar measure 0. Otherwise, please clarify what you mean by "a distribution consistent with Haar measure over $O(N)$".</p> <p>Individual entries of Haar-distributed $O(N)$ matrices are approximately Gaussian; in some form this goes back to Maxwell. This has recently been improved in two slightly different directions. First, fairly large submatrices are approximately distributed like matrices with i.i.d. Gaussian entries; see <a href="http://arxiv.org/abs/math/0601457" rel="nofollow">this paper by Jiang</a>. Second, linear combinations of the entries are approximately Gaussian, in a very strong sense; see <a href="http://arxiv.org/abs/math.PR/0509441" rel="nofollow">this paper by E. Meckes</a>. They are even approximately jointly Gaussian; see <a href="http://arxiv.org/abs/math.PR/0701464" rel="nofollow">this paper by Chatterjee and E. Meckes</a>.</p> <p><strong>Update:</strong> I discussed this question with Elizabeth, who pointed out to me that my suggestion isn't necessarily great because Gaussian behavior can creep in for all sorts of reasons, in particular due to an accumulation of small, nearly independent errors (i.e., the good old central limit theorem). But you the suggestion may be salvageable by getting more quantitative. For example, Elizabeth's paper shows that linear functionals approximate Gaussians with an error (in total variation) of $c/N$, which is better than one would expect from CLT effects. Even better, you can consider traces of powers, which were <a href="http://www.jstor.org/pss/2951843" rel="nofollow">shown by Johansson</a> to converge <em>exponentially</em> quicky to a Gaussian distribution.</p> http://mathoverflow.net/questions/77332/statistics-for-haar-measure-of-random-matrices/77402#77402 Answer by Carlo Beenakker for Statistics for Haar measure of random matrices? Carlo Beenakker 2011-10-06T21:45:42Z 2011-10-06T21:45:42Z <p>The ensemble of real orthogonal matrices (uniformly distributed with respect to the Haar measure) is the socalled Circular Real Ensemble (CRE) of random-matrix theory. (Not the Circular Orthogonal Ensemble, COE, which confusingly enough contains symmetric complex unitary matrices.)</p> <p>The probability distribution of the eigenvalues in the CRE is known, and you could use that to test whether your set of matrices is indeed drawn from that ensemble. (For example, by comparing moments of the eigenvalues or by comparing the spacing distribution.) </p> <p>You can find the eigenvalue probability distribution in Section 2.9.2 of P.J. Forrester's book "Log-gases and Random matrices". You will have to distinguish the cases of determinant equal to +1 or -1, and even or odd N. The formulas are a bit lengthy, but if the book is not available to you, let me know and I will try to record them here.</p> <p>Alternatively, you could just generate matrices from the CRE and compare the eigenvalue statistics of those matrices with your M matrices from an unknown ensemble. Generating matrices from the CRE is easy: They contain the eigenvectors of random real symmetric matrices with a Gaussian distribution of the matrix elements (the Gaussian Orthogonal Ensemble --- yes, here the word orthogonal is appropriate). </p>