Finite-maximal subgroups of orthogonal groups

Question

I define a finite subgroup $H$ of a group $G$, finite-maximal if for any $g\in G\setminus H$, $\langle H,g\rangle$ is infinite. My question is now to find the finite-maximal subgroups of $\mathrm{SO}_n(\mathbb R)$ and of $\mathrm{O}_n(\mathbb R)$. It is clear that if $n=2$, there are no finite-maximal subgroups. If $n=3$, we get the (full) octahedral and (full) icosahedral symmetry groups. If $n=4$ I think we get $[3,3,5]$ and $[[3,4,3]]$, but probably also what Conway calls $\pm[O\times I]$. Are these first cases correct? What about higher values of $n$?

Answer 1

The group $(\mathbb{Z}/2\mathbb{Z}) \wr S_{n}$ is finite maximal for $n$ sufficiently large (probably $n>72$ will do, by a theorem of M.Collins on a sharp form of Jordan's theorem on finite complex linear groups, and this is likely more generous than the true bound). I mean here the group of $n \times n$ monomial matrices with all non-zero entries $\pm 1$.

To roughly sketch out the argument for large enough $n$: think of the hyperoctahedral group as a complex linear group of degree $n$. If it is contained in larger finite subgroup X of ${\rm O}_{n}(\mathbb{R}),$ the (viewed as complex linear group), $X$ has an Abelian normal subgroup $A$ with $[X:A] \leq (n+1)!,$ and the only way any index greater than $n!$ (but less than or equal to $(n+1)!$ can be attained is if $X/A$ has a composition factor $A_{n+1}.$ But if $X/A$ has a composition factor $A_{n+1},$ then $X$ must be (irreducible and) primitive as complex linear group, in which case $A$ must consist of scalars. But since $X$ is really a subgroup of ${\mathbb{O}}_{n}(\mathbb{R})$, we have $|Z(X)| \leq 2$ and $|X| \leq (2n+2)n!,$ contrary to $|X| > 2^{n}n!.$ I should add that if $[X:A] \leq n!,$ we get $|A| \leq 2^{n},$ which is also a contradiction.

Later edit: as to the existence or otherwise of other finite maximal subgroups of ${\rm O}_{n}(\mathbb{R}),$ it is true that every non-Abelian finite simple group $G$ is isomorphic to an absolutely irreducible subgroup of ${\rm O}_{n}(\mathbb{R})$ for some $n$ (equivalently, every such simple group has a non-trivial irreducible complex character with Frobenius-Schur indicator $1$). We could consider (necessarily faithful) irreducible character of least degree, say $m$, with this property for $G$.

In this embedding, it might be that the representation extends (as real representation) to some part of the automorphism group of the simple group (that is, to a larger almost simple group with the same unique minimal normal subgroup)). In "most" cases, I would expect that the largest almost simple group to which it extends would be a finite maximal subgroup of ${\rm O}_{m}(\mathbb{R}).$