# Finite groups with few double cosets with respect to abelian subgroup

The following question is motivated by the study of certain tensor categories, namely integral near-group categories.

Let $G$ be a finite group and $H\subset G$ be a subgroup. Is it possible to give a complete classification (or at least give some examples) satisfying the following conditions:

1) $H$ is abelian;

2) $G$ is the union of the normalizer $N$ of $H$ and exactly one more double coset with respect to $H$.

Pairs $(G,H)$ satisfying the additional condition below is of special interest:

3) the double coset not contained in $N$ consists of $|H|^2$ elements (which is maximal possible size)

Known examples satisfying 1), 2) and 3):

a) $G=F_q\rtimes F_q^*$ and $H=F_q^*$ (here $F_q$ is a finite field)

b) $G=S_4$ and $H$ is a non-normal subgroup of order 4.

An easy observation is that condition 2) implies that $G-$action on $G/N$ is 2-transitive but I don't know how to use it.

-
If $S \subset H$ is a normal subgroup of $G$, Then $(G,H)$ satisfies the condition if and only if $(G/S,H/S)$ does. Condition 3 is equivalent to the condition that there is no nontrivial $S$, as if $HgH$ is the unique coset of size greater than $|H|$, but has size less than $|H|^2$, then $H \cap gH g^{-1}$ is such a subgroup. So we might as well assume this condition, in which case $G$ is a transitive permutation group acting on $G/H$ where the stabilizer of a point abelian. Then $|G/H| =|N/H|+|H|$, but $|N/H|$ divides $|G/H|$, so $|N/H|$ divides $|H|$. –  Will Sawin Sep 21 '13 at 20:30
If $|N/H|=1$ then $G$ is a Frobenius group. From this you can deduce that $H$ is cyclic and acts on a nilpotent group $K$ by conjugation, which fixes the identity and acts transitively on the rest. Thus $K$ can only be a $\mathbb F_p$-vector space, and since $H$ acts simply transitively it gives the nonzero vectors a group structure, making $K$ into a finite field. So example a) is the only example when $|N/H|$ is $1$ and condition $3$ is true. –  Will Sawin Sep 21 '13 at 20:43

We can construct such a group in the following way:

First take $\mathbb F_q \rtimes \mathbb F_q^{\times}$.

Then take a product of two isomorphic abelian groups $X \times Y$, where $X\cong Y$. (This notation appears strange but is more convenient.)

Choose an action of $\mathbb F_q \rtimes \mathbb F_q^{\times}$ on $X\times Y$ by automorphisms where $\mathbb F_q^{\times}$ fixes $X$ but some element sends $X$ to $Y$.

Then choose an extension $1 \to X \times Y \to G \to \mathbb F_q \rtimes \mathbb F_q^{\times}$ where previous action is the action by conjugation and the inverse image of $\mathbb F_q^{\times}$ has a normal subgroup $H$ containing $X$ with complement $Y$. $G,H$ is a pair of the type you desire. These are the only type of pair satisfying conditions 1 to 3.

The choice of extension is the only fiddly bit in terms of such a classification. I don't know how to classify these types of extensions - maybe by some type of group cohomology? However, extensions of $1 \to X \times Y \to N \to \mathbb F_q^{\times}$ can be easily classified - they consist of pairs of an extension $1 \to X \to H \to \mathbb F_q^\times$ (itself classified by maps $\mathbb F_q^{\times} \to X$) and an action of $Y$ on $H$ which fixes the components, which is just a map $Y \otimes \mathbb F_q^\times \to X$. But I don't know how many lifts each of these will have to the whole group.

But note that lots of easy extensions, like semidirect products, do satisfy these conditions!

To prove that this is actually a pair of the type you desire, first note that $H$ is an extension of a cyclic group by an abelian group where the abelian group is central, hence it is abelian. Then note that the order of $G$ is $q(q-1)|X|^2=|H|^2 +(q-1)|X|^2$, but the normalizer of $H$ contains the inverse image of $\mathbb F_q^{\times}$ so it has size at least $(q-1)|X|^2$, so we just need to find a double coset of size $|H|^2$. Let $g\in g$ be such that $gXg^{-1}=Y$. Then I claim $gHg^{-1} \cap H=1$, producing the coset. Since $g$ cannot be in the inverse image of $\mathbb F_q^{\times}$, the image of $H \cap gHg^{-1}$ in $\mathbb F_q \rtimes \mathbb F_q^{\times}$ is trivial, so any intersection must lie in $X \times Y$. But $H \cap X \times Y = X$, and $gXg^{-1}=Y$, and $X \cap Y=1$, so the intersection is trivial.

To prove that every pair satisfying conditions 1,2, and 3 can be described in this way, take the coset $HgH$ of size $|H|^2$. Since it can also be described as the complement of $N$, it is closed under the action of $N$, so $NgH$ has order $|H|^2$ as well. Let $X$ be the subgroup of $H$ which acts trivially on $Ng$ on the right, and let $Y$ be the subgroup of $N$ that acts trivially on $gH$ on the left. Then $X$ is conjugate to $Y$ by $g$, $|X|=|Y|=|N|/|H|$, and $Y \cap H=1$.

Since $Y\subset N$ and $X \subset H$, $Y$ normalizes $X$. Since $Y\cap X=1$, $XY$ is a subgroup of order $|N/H|^2$. Next we will check that $XY$ acts trivially on $G/N$. $X \subset H$, and $H$ acts on $G/N$ with a single nontrivial orbit, and is abelian, so any subgroup of it that fixes one point fixes every point, so $X$ acts trivially on $G/N$. $Y$ is conjugate to $X$, so $XY$ acts trivially on $G/N$.

Next we see that $XY$ is everything that acts trivially on $G/N$. Indeed, every element that acts trivially must be in $N$, but $|NgN|=|H|^2$, so only $|N|^2/|H|^2=|XY|$ elements can act trivially.

Now we check that $G/(XY)$ is a group of this type. Indeed, the image of $H$ in $G/(XY)$ is abelian, and the action of $H$ on $G/(XY)H= G/N$ is a single nontrivial orbit plus a single trivial orbit, so $G/XY$ is a group of this type and, moreover, a Frobenius group, so as I showed in my comment, it is $\mathbb F_q \rtimes \mathbb F_q^{\times}$.

Now we are done. $XY$, the product of two isomorphic abelian groups, is a normal subgroup with quotient $\mathbb F_q \rtimes \mathbb F_q^{\times}$, which acts by conjugation, with one element sending $X$ to $Y$, but $\mathbb F_q^{\times}$ fixing $X$, since they are both part of the abelian subgroup $H$. $H$ is a normal subgroup of $N$, containing $X$, and $Y$ is a complement, since $Y \cap H=1$ and $|Y|=|N|/|H|$.

-
Dear Will, thanks very much! This is very useful. –  Victor Ostrik Sep 22 '13 at 19:08
@VictorOstrik: Hopefully this stronger result will be a bit more useful! –  Will Sawin Sep 23 '13 at 0:18