Is {6,3,7} an 'ultrahyperbolic' Coxeter group?

Question

These pictures, drawn by Roice Nelson, are attempts to visualize a geometry having as symmetries the {6,3,7} Coxeter group, by which I mean the one coming from the Coxeter diagram

$$\circ-6-\circ-3-\circ-7-\circ$$

Does this group act on a 4-dimensional real vector space equipped with a quadratic form of signature $++--$, with the four standard generators acting as reflections?

More generally: it's easy to learn about Coxeter groups acting as reflections on real vector spaces with a quadratic form of signature $(n,1)$: these 'hyperbolic' Coxeter groups show up as symmetries of hyperbolic honeycombs, and there's a lot of literature available on those. But what about 'ultrahyperbolic' Coxeter groups, which act as isometries of real vector spaces with quadratic form of signature $(n,m)$ with $n, m \ge 2$? Where can we learn about those?

Here is what Roice Nelson wrote about the above pictures, as part of a discussion about a puzzle I posed on Google+:

I just uploaded two pictures of the {6,3,7} to wikipedia:

https://commons.wikimedia.org/wiki/File:H3_637_UHS_plane_at_infinity_view_1.png

https://commons.wikimedia.org/wiki/File:H3_637_UHS_plane_at_infinity_view_2.png

The images look nothing like the {6,3,6}, so here's a little about what's going on and why I drew them differently...

When Coxeter extended the enumeration of honeycombs from 4 to 15, he did so by allowing the fundamental simplex to have "ideal" vertices. But he still limited the volume of the simplex to be finite. This restriction is equivalent to requiring the cells and vertex figures to be spherical and/or Euclidean. For a {p,q,r} honeycomb, $(p-2)(q-2) \le 4$ and $(q-2)(r-2) \le 4$.

But we can extend the enumeration of honeycombs to be endless if we allow the fundamental simplex to have infinite volume (to be noncompact). This means that one or more vertices of the simplex disappear completely, living beyond the plane at infinity. I've seen such points described in the literature as "ultra-ideal". Points outside the hyperbolic models do have geometric meaning btw (see Thurston's Three Dimensional Geometry and Topology p. 71). Noncompact honeycombs have cells and/or vertex figures that are hyperbolic tilings.

In the progression above, note how the cells appear to fill more and more of hyperbolic space. For the first 3, we can see vertices inside H3, but for the {6,3,6}, the vertices have become ideal and each {6,3} cell is inscribed in the whole of hyperbolic space. Continuing to {6,3,7}, we might expect the vertices to continue their expansion, and indeed they do become ultra-ideal.

If we were to draw the {6,3,7} honeycomb in the same style, the edges would never meet, so we'd see a bunch of disconnected edges all over the place. Instead what the wiki images show is how the honeycomb interacts with the plane at infinity. It is a single slice of the honeycomb with that plane, in the upper half space model.

The vertex figure of a {6,3,7} honeycomb is a {3,7} tiling. What we see in the slice is a {3,7} tiling for every ultra-ideal vertex of the honeycomb. This arises because the cells haven't met at a vertex yet (even at infinity). Furthermore, we see these vertices laid out in {6,3} patterns. I've included two views, but it is the same honeycomb. The second may be easier to digest because the hexagonal pattern of vertices is more typical.

What would the {6,3,6} look like in a similar slice view? Well, those cells meet at ideal points on the boundary, and the points densely fill the plane at infinity, so it would just look solid black :)

The {6,3,7} puzzle is difficult in my eyes because honeycombs with {6,3} cells are already strange to get a handle on. Plus some formulas in Coxeter's paper are useless (e.g. the formula for in-radius already results in infinity for the {6,3,6}, and so doesn't help for {6,3,7}). Some easier noncompact honeycombs to think about first are the {3,3,7} and {7,3,3}.

Sorry, this comment is getting much longer than I wanted, and is probably opaque as dirt, so I think I'll stop for now. Henry Segerman and I have started preparing a math-art paper to share some of these images, and discuss this topic in more depth, hopefully more clearly!

I don't understand how all this relates to my conjecture that {6,3,7} acts on a 4d vector space with quadratic form of signature $++--$.

Answer 1

Actually, every Coxeter system of rank four is either euclidean or hyperbolic. That is, the canonical bilinear form has at most one negative eigenvalue.

It follows from the paper Sphere packings and hyperbolic reflection groups of Maxwell. He categorizes the Coxeter systems by "level": finite or affine Coxeter systems are of level 0 (some may be more comfortable with terms like "spherical/euclidean" etc.). Then a general Coxeter system is of level $\ell$, if deletion of any $\ell$ generators leaves a system of level $0$, while deletion of some $\ell-1$ generators leaves a system of level $>0$.

Then a theorem of Maxwell says:

Every Coxeter system of level $1$ or $2$ is hyperbolic.

It is easy to see that every Coxeter system of rank $4$ is of level at most $2$.

This is not true if we take the more general representation of Vinberg. In Vinberg's convention, $B(e_s,e_t)$ is allowed to be $<-1$. In the Coxeter diagram, such an edge is dashed, and labeled with a number $<-1$. If this happens to a Coxeter system of rank $4$, the system is no longer level $2$ in Maxwell's sense. It is then easy to construct an example of signature $(2,2)$: just take two copies of $\circ\cdots-2\cdots\circ$.

For systems of rank $\ge 5$, as part of a joint work with Labbé on limit roots, I recently enumerated all 326 Coxeter systems of level $2$, see http://arxiv.org/abs/1310.8608. This enumeration was done by Maxwell by hand, and he suggest a computer verification. My program finds three systems missed by him, and the list is double checked by Maxwell himself.

For Coxeter systems with signature $(m,n)$ with $m,n\ge 2$, you can find examples in http://arxiv.org/abs/1403.1502 (Example 3.11) and http://arxiv.org/abs/1303.6710 (Figure 5). Again, these examples are studied in the context of limit roots.

Answer 2

Suppose we have an action of the 6,3,7 reflection group on some nondegenerate real quadratic space by reflections in positive norm vectors. Then there is a 4-tuple of vectors for which the reflections in the perpendicular hyperplanes generate the group according to the Coxeter relations. We may assume these vectors have length 1, so the inner products are given by the Schäfli matrix $$\left( \begin{smallmatrix} 1 & -\cos(\pi/6) & 0 & 0 \\ -\cos(\pi/6) & 1 & -\cos(\pi/3) & 0 \\ 0 & -\cos(\pi/3) & 1 & -\cos(\pi/7) \\ 0 & 0 & -\cos(\pi/7) & 1 \end{smallmatrix} \right).$$

According to SAGE, this bilinear form has signature $(3,1)$, so the span of these vectors contains a three-dimensional positive-definite subspace. Such a subspace does not exist in $\mathbb{R}^{2,2}$.

Answer 3

In fact, it seems that the Coxeter group ∘−6−∘−3−∘−7−∘ is not 'ultrahyperbolic', but hyperbolic. That is, this group is a subgroup of the Lorentz group. In other words, it's a group of linear isometries of $\mathbb{R}^{3,1}$.

To see this we use the fact that Qiaochu Yuan mentioned: any Coxeter group coming from a diagram with $n$ dots acts as linear isometries of $\mathbb{R}^n$ equipped with the symmetric bilinear form $g$ whose matrix is

$$ g_{ij} = -2 \cos(\pi/ m_{ij}). $$

Here $m_{ij}$ is the number labelling the edge from the $i$th dot to the $j$th, or $2$ if there's no edge shown and $i \ne j$, or $1$ if $i = j$.

In our example, $g_{ij}$ is twice the matrix given by Scott Carnahan:

$$2 \left( \begin{smallmatrix} 1 & -\cos(\pi/6) & 0 & 0 \\ -\cos(\pi/6) & 1 & -\cos(\pi/3) & 0 \\ 0 & -\cos(\pi/3) & 1 & -\cos(\pi/7) \\ 0 & 0 & -\cos(\pi/7) & 1 \end{smallmatrix} \right).$$

The factor of $2$ doesn't matter much here: what matters is that the signature is $(3,1)$, just as Scott said. And this means---maybe Scott was trying to tell me this, but I was too thick to realize it---that the Coxeter group corresponding to ∘−6−∘−3−∘−7−∘ acts as linear isometries of $\mathbb{R}^{3,1}$.

Greg Egan checked this with his own calculation. In the same way, he showed that the Coxeter group corresponding to ∘−7−∘−3−∘−3−∘ is a subgroup of the Lorentz group. This group acts as symmetries of a structure shown in another charming picture by Roice Nelson: