I had a go at this question, but the method I tried here doesn't quite work out. It does reduce it to upper triangular matrices, although that doesn't seem to be a lot of help for general n.

Let your volume be V.

By scaling, the volume of the set {|A|&le;K} is VK<sup>n<sup>2</sup></sup>. Now let M be a matrix whose entries are independent normal random variables with mean 0 variance 1. From the density function of the normal distribution, this gives P(|M|&le;K)~(2&pi;)<sup>-n<sup>2</sup>/2</sup>VK<sup>n<sup>2</sup></sup> in the limit of small K.

I'll now calculate this expression in an alternative way. Use the M=QR decomposition, where Q is orthogonal and R is upper triangular, with diagonal elements &lambda;<sub>n</sub>, &lambda;<sub>n-1</sub>,&hellip;&lambda;<sub>1</sub>, which are the eigenvalues of R. This can be done in such a way that &lambda;<sub>k</sub><sup>2</sup> has the <a href="http://en.wikipedia.org/wiki/Chi%5E2">&chi;<sup>2</sup><sub>k</sub>-distribution</a> (a quick google search gives <a href="http://www.nowpublishers.com/product.aspx?product=CIT&doi=0100000001&section=x1-36r1">this</a> but there's probably better references). The upper triangular parts of R have the standard normal density. We need to calculate |R|. I was originally thinking that this is the max eigenvalue, but it's not quite that simple.