Let $G$ be a finite subgroup of $\mathrm{SO}_n(\mathbb R)$. We also assume $G$ to be "maximal" in the sense that for every $g\in\mathrm{SO}_n(\mathbb R)\setminus G$, we have that $\overline{\langle G,g\rangle}=\mathrm{SO}_n(\mathbb R)$. Let $H:=\{h\in\mathrm{O}_n(\mathbb R)\setminus\mathrm{SO}_n(\mathbb R): |\langle G,h\rangle|<\infty\}$. Is it true that for any $h,h'\in H$, if $h\overset{\mathrm{O}_n(\mathbb R)}\sim h'$, then $\langle G,h\rangle\overset{\mathrm{O}_n(\mathbb R)}\sim\langle G,h'\rangle,$ where $\overset{\mathrm{O}_n(\mathbb R)}\sim$ is the conjugate relation in $\mathrm{O}_n(\mathbb R)$? Notice that for any $h\in H$, $|\langle G,h\rangle|=2|G|$, because $G=\ker\det$ is a normal subgroup of $\langle G,h\rangle$ of index $2.$
-
3$\begingroup$ Your assertion that $h\in H$ implies $|\langle G,h\rangle|=2|G|$ is incorrect. Indeed $\langle G,h\rangle\cap\mathrm{SO}_n$ can be larger than $G$ (pick $G=\{1\}$ to find the most trivial counterexamples). Or add the condition $\langle G,h\rangle\cap\mathrm{SO}_n=G$ in your definition of $H$. $\endgroup$– YCorCommented Oct 13, 2023 at 15:12
-
$\begingroup$ Thank you @YCor. The maximality assumption is essential and I now moved it. $\endgroup$– Andrea AveniCommented Oct 13, 2023 at 16:23
1 Answer
Now that you've added the maximality assumption, $\langle G,h\rangle$ has to be the normaliser of $G$ in $\mathop{\rm O}_n(\mathbb{R})$, and so does $\langle G,h'\rangle$. So they are not just conjugate, but equal. The set $H$ is the other coset of $G$ in its normaliser.
Here's why: Since $\langle G,h\rangle$ is finite, so is $\langle G,G^h\rangle$. This is then a finite subgroup of $\mathop{\rm SO}_n(\mathbb{R})$ containing $G$, and therefore by maximality we have $G^h=G$, so $h$ normalises $G$.
To expand the argument a bit, the centraliser of $G$ in $\mathop{\rm SO}_n(\mathbb{R})$ is $Z(G)$, since otherwise $G$ can be enlarged to a larger finite subgroup. So the centraliser in $\mathop{\rm O}_n(\mathbb{R})$ is also finite. The normaliser mod the centraliser is a group of automorphisms of a finite group, so is finite. Therefore the normaliser is finite.
-
$\begingroup$ I agree that $G^h=G$ and thus $\langle G,h\rangle$ is a subgroup of the normalizer of $G$ in $\mathrm O_n(\mathbb R)$. I also agree that if we assume this normalizer to be finite, then $\langle G,h\rangle=\langle G, h'\rangle$ (because otherwise $hh'$ would be in $\mathrm{SO}_n(\mathbb R)\setminus G$ and, with $G$ it would generate an infinite group). But why should this normalizer be finite? Thank you $\endgroup$ Commented Oct 13, 2023 at 19:04
-
2$\begingroup$ For index reasons: restrict the determinant to the normalizer $N$; then its kernel must be $G$, and by the homomorphism theorem the kernel has index 2. $\endgroup$– Max HornCommented Oct 13, 2023 at 19:17
-
$\begingroup$ Sorry, but why is that $\ker \det|_{N_{\mathrm{O}_n(\mathbb R)}(G)}=G$ and not some dense subgroup of $\mathrm{SO}_n(\mathbb R)$? I guess this equality holds iff $N_{\mathrm{SO}_n(\mathbb R)}(G)$ is finite, but I don't see why this must be the case. $\endgroup$ Commented Oct 13, 2023 at 20:47
-
2$\begingroup$ The normalizer $N$ of $G$ in $\mathrm{SO}_n$ is reduced to $G$. The reason is that $N/G$ being a compact Lie group, if it is not trivial, then it has a nontrivial finite subgroup, and hence pulling back contradicts the maximality of $G$ among finite subgroups of $\mathrm{SO}_n$. Hence either $H$ is empty and $G$ equals its normalizer in $\mathrm{O}_n$, or $H$ is nonempty and $G$ has index two in its normalizer, which is then $G\sqcup H$. $\endgroup$– YCorCommented Oct 13, 2023 at 23:03
-