# Is a prime index inclusion of finite groups, separating?

Let $(H \subset G)$ be an inclusion of finite groups.
Let $\{ g_i \ \vert \ i \in I=[1, \dots ,n] \}$ a subset of $G$ of double coset representatives, i.e. $$G = \coprod_{i \in I} Hg_iH$$ On the algebra $\mathbb{C}G$ we consider the elements $$e_i = \frac{1}{\vert H \vert} \sum_{g \in Hg_iH} g$$ They generate the double coset subalgebra of $\mathbb{C}G$, we have the following constants structure $c_{i j}^{k} \in \mathbb{N}$: $$e_{i} \cdot e_{j} = \sum_{k \in I} c_{i j}^{k} e_{k}$$ Remark: we get the first examples of finite hypergroups (up to a renormalisation), noted $G//H$.

For example, with $(H \subset G) = (\mathbb{Z}/2 \subset \mathbb{Z}/5 \rtimes \mathbb{Z}/2)$ we get the following table (with $g_1 = e$ the neutral):
$$\begin{array}{c|c|c|c|c|c|c} \cdot & e_1 & e_2 & e_3 \\ \hline e_1 & e_1 & e_2 & e_{3} \\ \hline e_2 & e_2 & 2e_1+e_3 & e_2+e_3 \\ \hline e_{3} & e_{3} & e_{2}+e_3 &2e_1+e_2 \end{array}$$

Definition: a basic element $e_{i}$ is called generating if $\forall j \in I, \exists r_j$ such that $(e_{i}^{r_j}, e_j) >0$.
Remark: It exists a generating basic element iff $\exists g \in G$ such that $\langle H,g \rangle =G$.
It is true for every primitive inclusion (and also for every "distributive" inclusion, see here).

Let the matrix $M_i = (m_{ij}^{(r)})_{rj}$ defined by $e_{i}^{r} = \sum_{j \in I}m_{ij}^{(r)}e_j$.

For the example above: $M_1 = \left(\begin{matrix} 1 & 0 & 0 \\ 1 & 0 & 0 \\ 1 & 0 & 0\end{matrix}\right)$, $M_2 = \left(\begin{matrix} 0 & 1 & 0 \\ 2 & 0 & 1 \\ 0 & 3 & 1\end{matrix}\right)$ and $M_3 = \left(\begin{matrix} 0 & 0 & 1 \\ 2 & 1 & 0 \\ 0 & 1 & 3\end{matrix}\right)$

Definition: a basic element $e_{i}$ is called separating if the matrix $M_i$ is invertible.
Remark: "separating" implies "generating", but the converse is false.
Definition: $(H \subset G)$ is called separating if it exists a separating basic element $e_{i}$.

Question: Is a prime index inclusion of finite groups, separating?
Remark: It's checked by GAP for $[G:H] \le 500$ and $\vert G \vert \le 4000$.

The first non-separating primitive inclusions are given by:

gap> G:=PrimitiveGroup(d,r);
gap> H:=Stabilizer(G,1);

with $(d,r) = (16,1), (16,2), (25, 1 ),( 25, 4 ),( 25, 6 ),( 25, 11 ), \dots$

For example with $(d,r) = (16,1)$ we get the following table and $M_i$ matrices:
$$\begin{array}{c|c|c|c|c|c|c} \cdot & e_1 & e_2 & e_3 & e_4 \\ \hline e_1 & e_1 & e_2 & e_{3} &e_4 \\ \hline e_2 & e_2 & 5e_1+2e_3+2e_4 & 2e_2+2e_3+e_4 & 2e_2+e_3+2e_4 \\ \hline e_3 & e_3 & 2e_2+2e_3+e_4 &5e_1+2e_2+2e_4 & e_2+2e_3+2e_4 \\ \hline e_4 & e_4 & 2e_2+e_3+2e_4 &e_2+2e_3+2e_4 & 5e_1+2e_2+2e_3 \end{array}$$ $M_1 = \left(\begin{smallmatrix} 1 & 0& 0& 0 \\ 1 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0\end{smallmatrix}\right)$, $M_2 = \left(\begin{smallmatrix} 0 & 1& 0& 0 \\ 5 & 0 & 2 & 2 \\ 0 & 13 & 6 & 6 \\ 65 & 24 & 44 & 44\end{smallmatrix}\right)$, $M_3 = \left(\begin{smallmatrix} 0& 0& 1& 0 \\ 5& 2& 0& 2 \\ 0& 6& 13& 6 \\ 65& 44& 24& 44 \end{smallmatrix}\right)$, $M_4 = \left(\begin{smallmatrix} 0& 0& 0& 1 \\ 5& 2& 2& 0 \\ 0& 6& 6& 13 \\ 65& 44& 44& 24\end{smallmatrix}\right)$

Bonus question: How to improve the "separating" assumption for becoming true for every primitive inclusion?
Idea to check: widen the notion to the existence of a separating (non-necessarily basic) element.

• It is very difficult to make out what all of this means. I think that the questioner is looking at the subalgebra of the complex group algebra of G spanned by the sums of the elements in the various H-H double cosets of G. I can't figure out, however, what the matrices M_i are supposed to be, and I don't understand the *r exponent notation. Can the questioner clarify this? – Marty Isaacs Jun 4 '15 at 20:53
• @MartyIsaacs: $e_i^{*r} = e_i * e_i * \dots * e_i$ ($r$ times). The matrix $M_i$ is invertible iff the set $\{e_i , e_i^{*2}, \dots, e_i^{*n} \}$ is a basis, – Sebastien Palcoux Jun 5 '15 at 5:16
• This is how I understood the *r-exponent notation, but I agree with @MartyIsaacs that it is slightly misleading, since (first) you can realize the algebra as a non-unital subalgebra of the group algebra and use the multiplication of the group algebra (so no need for $*$), and (second) since the group algebra and the double coset subalgebra has a natural involution which is often denoted by $a\mapsto a^*$ (sending $g$ to $g^{-1}$. – Frieder Ladisch Jun 5 '15 at 13:53
• @FriederLadisch: $a*b$ can be interpreted as a convolution (it is the usual notation), but you are right this interpretation is useless here and I will try to simplify the notations according to your comment. – Sebastien Palcoux Jun 5 '15 at 14:23
• The notation has been improved, there is no more $*$. – Sebastien Palcoux Jun 5 '15 at 14:57

I think the answer for prime index inclusions is Yes. I see the question from the following point of view: For each double coset $D= HgH$, you form the element $$e_D := \frac{1}{\lvert H \rvert} \sum_{x\in D} x \in \mathbb{Q} G .$$ The product of two such elements is an $\mathbb{N}$-linear combination of such elements. It follows that the $e_D$'s form the basis of a $\mathbb{Q}$-algebra $C$ (say), and $e_H$ is the $1$ of this algebra. This algebra $C$ is isomorphic to the centralizer algebra of the corresponding permutation representation (over $\mathbb{Q}$).
(Later edit: That is, the centralizer of the permutation matrices in the ring of quadratic $\mathbb{Q}$-matrices of size $\lvert G:H\rvert$. Possibly the most elegant way to see this is to realize the permutation module as the left ideal $\mathbb{Q}G e_H$ with $G$ acting from the left. Then the centralizer can be identified with the endomorphism ring $\operatorname{End}_{\mathbb{Q}G} (\mathbb{Q}G e_H)$, which is isomorphic to the ring $e_H \mathbb{Q}G e_H$, acting from the right. And the $e_D$'s (with $D$ running over the $H$-$H$-double cosets) form a basis of $e_H \mathbb{Q}Ge_H$.)

The rows of a particular matrix $M_i$ contain the coefficients of $e_D$, $(e_D)^2$, $\dots$, $(e_D)^n$ in terms of the natural basis of $C$, where $D = Hg_iH$. Thus $e_D$ is a separating element if and only if the powers of $e_D$ form a basis of the centralizer ring $C$. This is the case if and only if $C= \mathbb{Q}[e_D]$ and $e_D$ is invertible. In particular, if there is a separating element (maybe not basic), then the centralizer ring must be commutative, or equivalently, the permutation representation must be multiplicity free. Conversely, if this is the case, then the centralizer ring is a direct product of fields, and at least some (non-basic) separating element exists.

Now let $G$ be a permutation group of prime degree $p$. By a theorem of Burnside, $G$ is either $2$-transitive, or is isomorphic to a proper subgroup of the affine group $\operatorname{AGL}(1,p)$. The case of $G$ being $2$-transitive is clear. In the other case, we may assume that $G$ is a subgroup of $\operatorname{AGL}(1,p)$, viewed as permutation group on $\mathbb{F}_p$, and $H$ is the stabilizer of $0_{\mathbb{F}_p}$. Let $\widehat{H}$ be the stabilizer in $\operatorname{AGL}(1,p)$ of $0_{\mathbb{F}_p}$. Then the $e_D$'s with $D$ not the double coset $H$ get permuted transitively under the conjugation action of $\widehat{H}$.

On the other hand, the centralizer algebra is isomorphic to the sum of $\mathbb{Q}$ and another field $\mathbb{K}$ contained in the cyclotomic field $\mathbb{Q}(\varepsilon_p)$. The group $\widehat{H}$ acts on $\mathbb{K}$ as field automorphisms. Let $\alpha$ be the projection of one $e_D$ onto $\mathbb{K}$, where $D\neq H$. It follows from the previous paragraph that the Galois conjugates of $\alpha$ generate $\mathbb{K}$. As all subfields of $\mathbb{K}$ are Galois, we see that $\mathbb{K} = \mathbb{Q}(\alpha)$. The projection of $e_D$ onto $\mathbb{Q}$ is non-zero (namely, $\lvert D \rvert / \lvert H \rvert$). Thus $e_D$ is an invertible element of the centralizer algebra. Thus to see that $e_D$ is separating, it suffices to show that $(e_D)^0 = e_H$, $e_D$, $(e_D)^2$, $\dots$, $(e_D)^{n-1}$ are linear independent over $\mathbb{Q}$. Suppose $\sum_{r=0}^{n-1} q_r (e_D)^r = 0$ with $q_r\in \mathbb{Q}$. Then the minimal polynomial of $\alpha$ must divide $p(x) = \sum_r q_r x^r$. The minimal polynomial of $\alpha$ has degree $n-1$ and $p(x)$ at most degree $n-1$, however, so $p(x)$ is the minimal polynomial up to a scalalr multiple (possibly zero). The projection of $e_D$ onto $\mathbb{Q}$ can not be a zero of the minimal polynomial of $\alpha$. Thus $p(x) = 0$. It follows that the centralizer ring is generated by the powers of $e_D$, so $e_D$ is separating.

• I try to understand why $C$ should be a centralizer (i.e. commutant). It should be the commutant of which subalgebra in which algebra? – Sebastien Palcoux Jun 5 '15 at 14:25
• @SébastienPalcoux: I edited my post to explain why $C$ is a centralizer. I hope it is clear now? – Frieder Ladisch Jun 5 '15 at 14:52
• I have to understand all your proof before accepting it. $AGL(1,p) = \mathbb{Z}/p \rtimes \mathbb{Z}/(p-1)$, right? – Sebastien Palcoux Jun 7 '15 at 10:45
• So in general with $H \subset G$ core-free, if $L_H$ is the representation of $G$ by left action on $L^2(G/H)$, then the commutant $L_H(G)'$ is isomorphic to the double coset algebra acting on the right. It's a very interesting generalization of the more-known result in the trivial case $H=\{ e \}$, and it seems that the trivial case can be used for getting a proof by contradiction for the general case. Is there a detailed proof of that in a book? – Sebastien Palcoux Jun 7 '15 at 12:46
• There also is a proof of the isomorphism between the commutant and the double coset algebra in the book Harmonic analysis of finite groups of Ceccherini-Silberstein, Scarabotti and Tolli (Proposition 4.2.1), which uses the language of the group algebra and the double coset algebra as function algebras with convolution as multiplication (the double coset algebra corresponds to $H$-$H$-bi-invariant functions on $G$). Maybe you preferthis proof. The statement is true whether or not $H\subseteq G$ is core-free, by the way. – Frieder Ladisch Jun 7 '15 at 18:35