A well-known theorem of Coxeter states that any discrete group $W$ which is generated by reflections across (possibly affine) hyperplanes in Euclidean space is a *Coxeter group*: it has a presentation with generators $\{r_i, i \in I\}$ and relations $r_i^2=e, \forall i$ and $(r_ir_j)^{m_{ij}}=e$ for some numbers $m_{ij} \in \{2,3,\ldots\} \sqcup \{\infty\}$.

I am interested in when the discreteness hypothesis can be discarded. For example, take the simplest indiscrete case: let $W$ be generated by reflections across two central hyperplanes meeting at a $\pi$-irrational angle. Then $W$ is not discrete since it contains a dense set of rotations, but $W$ is still a Coxeter group with generators $\{r_1,r_2\}$ corresponding to the two given reflections and $m_{12}=\infty$.

Note that I do not require $W$ to be generated by finitely many reflections, nor my Coxeter groups to have $|I|$ finite. However I would still be interested to know about any results particular to the finite case.

Questions:

- Is any such group $W$ a Coxeter group? I suspect not.
- As a particular example: is the orthogonal group a Coxeter group? Again, probably not, but I don't see how to prove this.
- Is there some nice condition, more general than discreteness, which guarantees that $W$ is a Coxeter group?

This other question also asks for reflection groups that aren't Coxeter groups, but without focusing on discreteness. It also doesn't insist on working in Euclidean space and does assume finite $|I|$. It got no answers.

5more comments