Does such an infinite index subgroup exist?

Question

Notation: If $G$ is a countable group and $H$ is a subgroup, for $g\in G$, let $|\mathcal{O}_{gH}|$ be the size of the $H$-orbit of $gH$ in the $H$-set $G/H$.

Does there exist a countable group $G$ and a subgroup $H$ with $[G\colon H]=\infty$ such that:

1. There is a $g\in G$ with $|\mathcal{O} _{gH}|\neq |\mathcal{O} _{g^{-1}H}|$,
2. $|\mathcal{O} _{gH}|=\infty$ if and only if $|\mathcal{O} _{g^{-1}H}|=\infty$, and
3. There is a constant $M>0$ such that $\displaystyle \frac{|\mathcal{O} _{gH}|}{|\mathcal{O} _{g^{-1}H}|} \leq M$ for all $g\in G$ with $|\mathcal{O} _{gH}|,|\mathcal{O} _{g^{-1}H}|\neq \infty$.

For instance, 2 and 3 above would be satisfied if:

• If $|\mathcal{O} _{gH}|\neq |\mathcal{O} _{g^{-1}H}|$, then $0< |\mathcal{O} _{gH}|,|\mathcal{O} _{g^{-1}H}|\leq M$ for some $M>1$ independent of $g$.

Answer 1

The number of elements in ${\mathcal O}_{gH}$ is the index of $H\cap gHg^{-1}$ in $H$. Your condition 1 means that there is an element $g$ such that $|H:H\cap gHg^{-1}|\lt |H:H\cap g^{-1}Hg|=|gHg^{-1}: H\cap gHg^{-1}|$. So you want $H\cap gHg^{-1}$ to have different but finite indices in $H$ and in $gHg^{-1}$. The best way to achieve what you want (including 2 and 3) is to consider a group $H$ with two isomorphic subgroups $U,V$ of different finite indices, then consider the HNN extension $\langle H,g\mid gUg^{-1}=V\rangle$.

There are many such groups. For example, the Thompson group $F$ has a subgroup of index 2 that is isomorphic to itself (see this paper, Corollary 3.3, where all subgroups of finite index in $F$ which are isomorphic to $F$ are described.

Another source of examples is the lamplighter group $L=\mathbb{Z}_2\wr \mathbb{Z}$. If $a$ is the generators of $\mathbb{Z}_2$ and $b$ is the generator of $\mathbb{Z}$, then $L_k=\langle a, b^k\rangle$ is isomorphic to $L$ and has a finite index in $L$. The isomorphism from $L$ to $L_k$ is $a\mapsto a$, $b\mapsto b^k$ (here $k\ge 1$).

I think it is worthwhile to check whether properties 2 and 3 holds for these examples.

Warning I will leave the rest of the previous answer as an illustration of the fact that everybody should remember to use multiplication table properly. Thanks to Ian Agol and Dave Penneys for pointing out my error.

For example, take $H=F_2\times F_3$, $U=U_1\times U_2$ where $U_1$ is of index 10 in $F_2$, $U_2$ is of index 12 in $F_3$ (in that case, by Schreier's formula, the rank of $U_1$ is 11, the rank of $U_2$ is 25), $V=V_1\times V_2$ with $V_1$ of index 24, $V_2$ of index $5$ (then the rank of $V_1$ is 25 and the rank of $V_2$ is 11). In that case $U$ is isomorphic to $V$. The other properties should follow from the general properties of HNN extensions.

Answer 2

I believe (1 and 2) and (3) are mutually exclusive. Here is a proof:

First, the commensurator $$ Comm_G(H) = \{g\in G : |\mathcal{O} _{gH}|, |\mathcal{O} _{g^{-1}H}|<\infty\} $$ is a group. We will show:

Lemma: $\varphi\colon Comm_G(H)\to \mathbb{Q}_{>0}$ by $g\mapsto \displaystyle\frac{[H\colon H\cap gHg^{-1}]}{[H\colon H\cap g^{-1}Hg]}$ is a homomorphism.

From the lemma, if we assume there is a $g\in Comm_G(H)$ such that $\varphi(g)=x>1$ (criteria 1 and 2), then the order of $g$ must be infinite, since $x>1$ implies $x^n>1$ for all $n\geq 1$. Since the order of $g$ is infinite, criterion 3 cannot hold since eventually $\varphi(g^n)=\varphi(g)^n=x^n>M$ for any $M>0$.

Proof of the lemma:

We must show $\varphi(g_1g_2)=\varphi(g_1)\varphi(g_2)$. Define the following constants:

• For $i=1,2$, $a_i = [H\colon H\cap g_iHg_i^{-1}]$ and $b_i = [H\colon H\cap g_i^{-1}Hg_i]=[g_iHg_i^{-1}\colon H\cap g_iHg_i^{-1}]$
• $a=[H\colon H\cap (g_1g_2)H(g_1g_2)^{-1}]$ and $b=[(g_1g_2)H(g_1g_2)^{-1}\colon H\cap (g_1g_2)H(g_1g_2)^{-1}]$

Note that since $x\mapsto g_1xg_1^{-1}$ is an automorphism of $G$, we have:

• $a_2=[g_1Hg_1^{-1}\colon g_1Hg_1^{-1}\cap (g_1g_2)H(g_1g_2)^{-1}]$ and $b_2=[(g_1g_2)H(g_1g_2)^{-1}\colon g_1Hg_1^{-1}\cap (g_1g_2)H(g_1g_2)^{-1}]$

Now look at the subgroup $K=H\cap g_1 Hg_1^{-1}\cap (g_1g_2)H(g_1g_2)^{-1}$, and define

• $a_1'=[H\cap (g_1g_2)H(g_1g_2)^{-1}\colon K]$
• $a_2'=[H\cap g_1Hg_1^{-1}\colon K]$
• $b_1'=[g_1Hg_1^{-1}\cap (g_1g_2)H(g_1g_2)^{-1}\colon K]$

which are all finite, since if we have a quadrilateral of groups $L_1\cap L_2\subset L_1,L_2\subset G$, we must have $[L_1\colon L_1\cap L_2] \leq [G\colon L_2]$. Now since index is multiplicative, we have

• $a a_1'=a_1a_2'$
• $ba_1' = b_2 b_1'$
• $a_2b_1'=b_1a_2'$

Solving for $a$ and $b$, we get $$ \frac{a}{b} = \frac{a_1a_2'}{a_1'}\frac{a_1'}{b_2b_1'}=\frac{a_1a_2'}{b_2b_1'}. $$ Now note that $\displaystyle \frac{a_2'}{b_1'}=\frac{a_2}{b_1}$, so we have $$ \varphi(g_1g_2)=\frac{a}{b} = \frac{a_1}{b_2}\frac{a_2}{b_1}= \frac{a_1}{b_1}\frac{a_2}{b_2}=\varphi(g_1)\varphi(g_2). $$