# Geometry of numbers for three by three matrices?

While trying to use Minkowski's theorem to calculate the (left) class number of a noncommutative ring, I ran into the following problem:

What is the volume of the largest symmetric convex subset $S$ of $\mbox{Mat}_3(\mathbb{R})$ such that every matrix in this subset has determinant at most $1$? In particular, is there such a set with volume greater than 885?

The best subset that I have been able to find so far is

$S_1 = \{M\in\mbox{Mat}_3(\mathbb{R})\mid \forall O\in O(3,\mathbb{R}), \mbox{ Tr}(MO) \le 3\}$.

If we apply Gram-Schmidt to the rows of $M$ and use the resulting orthonormal basis for the columns of an orthogonal matrix $O$, then by the AM-GM inequality we can conclude that $\det M \le 1$ from $\mbox{Tr}(MO)\le 3$. Furthermore, if $S$ is any symmetric convex set of matrices of determinant at most $1$ which contains an orthogonal matrix $O$, then every $M\in S$ must satisfy $\mbox{Tr}(MO^{-1}) \le 3$ by considering the tangent hyperplane to the collection of matrices with determinant $1$. Thus $S_1$ is the largest such set which contains all orthogonal matrices.

Unfortunately, it's incredibly difficult to calculate the volume of $S_1$. All I've been able to do so far is to note that it contains the set $S_r$ of matrices such that the sum of the norms of the rows is at most $3$, and that the volume of $S_r$ is $\frac{972\pi^3}{35}\approx 861.089$. If we let $S_c$ be the similar set obtained by considering the columns, then Monte-Carlo integration indicates that the volume of $S_r\cup S_c$ is around $1050$, but I haven't been able to prove this.

What is the volume of $S_1$? If this is too hard, what is the volume of $S_r\cup S_c$?

## Motivation

The ring I am dealing with is $\mathbb{Z}\langle a,x\rangle/(a^3 = a-1, x^3 = 3x-1, xa = a(x^2-2)+2-x)$. It is an order in the central simple algebra over $\mathbb{Q}$ with invariants $1/3$ at $3$, $2/3$ at $2$, and $0$ at every other prime. The discriminant of this ring is $-6^6$ (as one can verify by working out the trace form by hand).

Since the associated central simple algebra has nontrivial invariants at $2$ and $3$, and since $\mbox{Nm}(x+1) = -3$ and $(a+xa-1)^3 = 2$, all left ideals of norm at most $4$ are principal. If we tensor this ring up to $\mathbb{R}$ it becomes $\mbox{Mat}_3(\mathbb{R})$, so we can think of it as a lattice in $\mbox{Mat}_3(\mathbb{R})$ of covolume $\sqrt{6^6}$. By Minkowski's bound, every left ideal class contains a representative of norm at most $\left(\frac{2^9\sqrt{6^6}}{\mbox{Vol}(S)}\right)^{1/3}$, so if we can find a set $S$ with volume at least $885$ then we can conclude that this ring has left class number $1$.

-
Why is $\left| \det M \right| \leq 1$ for all $M \in S_1$? It's not immediately obvious to me (though I have the feeling it should be easy). – Noam D. Elkies Jul 6 '13 at 21:07

The volume of $S_1$ is $\frac{16767 \pi^4}{560} \approx 2916.53$.
The idea is to realize that the set $S_1$ can be equivalently described as the nuclear norm (sum of the singular values $\sigma_i$) of $M$ being less than or equal to 3, i.e., $S_1 = \{M : \sum_i \sigma_i(M) \leq 3 \}$.
This is an orthogonally invariant set, so from the singular value decomposition $M = U \Sigma V^T$ we can do the integral pretty explicitly, using the fact that the Jacobian is given by $(\sigma_1^2-\sigma_2^2) (\sigma_1^2-\sigma_3^2)(\sigma_2^2-\sigma_3^2)$, and the integral is over the polytope $\{(\sigma_1,\sigma_2,\sigma_3) : \sigma_1 \geq \sigma_2 \geq \sigma_3 \geq 0, \, \sigma_1 + \sigma_2 + \sigma_3 \leq 3\}$ (this integral is $\frac{16767}{17920}$). The other factor corresponds to the orthogonal matrices $U,V$, and is $\frac{1}{2^n} Vol(O^n)^2= \frac{1}{2^n} (\prod_{i=1}^n Vol(S^{i-1}))^2 = \frac{1}{2^n} (\prod_{i=1}^n \frac{2 \pi^{i/2}}{\Gamma(i/2)})^2$, which for $n=3$ gives $32 \pi^4$.
@Noam, an easy way to see this is $|\det M | = \prod_i \sigma_i \leq (\frac{1}{n}\sum_i \sigma_i)^n$. – coma Jul 6 '13 at 23:08