Assume everything is finite.

Let $G$ be a primitive permutation group with point stabiliser $G_\alpha$ for some $\alpha$. For $\beta\ne\alpha$, by an arc stabiliser we mean $G_{\alpha\beta}=G_\alpha\cap G_\beta$ and an edge stabiliser we mean $G_{\{\alpha,\beta\}}$, the stabiliser of the set $\{\alpha,\beta\}$. Note that $G_{\{\alpha,\beta\}}\ge G_{\alpha\beta}$.

My question is: can an arc stabiliser $G_{\alpha\beta}\ne 1$ be normal in $G_\alpha$?

This is impossible if $G_{\{\alpha,\beta\}}> G_{\alpha\beta}$. In this case, there exists $g\in G$ such that $\alpha^g=\beta$ and $\beta^g=\alpha$. It follows that $g\in N_G(G_{\alpha\beta})$. Note that $G_\alpha$ is maximal in $G$ and $G_{\alpha\beta}$ cannot be normal in $G$ (otherwise $G_{\alpha\beta}$ will be in the kernel of this action), the normaliser $N_G(G_{\alpha\beta})=G_\alpha$. This gives a contradiction: $g\in G_\alpha$ but $\alpha^g=\beta\ne \alpha$.

I think there might exist an example in the case when $G_{\{\alpha,\beta\}}=G_{\alpha\beta}$. However, by a quick check with Magma there is no such primitive permutation group $G$ with $|G|\le 300$.

An equivalent statement is: Let $G$ be a group and $H$ a maximal and core-free subgroup of $G$. Is it possible that $1\ne H\cap H^g\lhd H$ for some $g\notin H$?

As shown in the comments there by Verret and Holt there is no such example for degree $\le 4095$. I also think the proof or an example, if any, will not be elementary (for example, applying O'Nan-Scott Theorem).

This was initially a MSE question I asked.

  • 2
    $\begingroup$ @GeoffRobinson I think that remark is essentially equivalent to $G_{\{\alpha,\beta\} }> G_{\alpha\beta}$. In the language of permutation groups, this is saying that the orbit of $\alpha^g$ under $G_\alpha$ is not self-paired. I conjecture that the answer to the question is no, but that the proof will require CFSG. (The question reminds me of the Sims Conjecture that, for a primitive group $G$, $|G_\alpha|$ is bounded as a function of $|\beta^{G_\alpha}|$ for any $\beta \ne \alpha$, which was evenutally proved using CFSG.) $\endgroup$
    – Derek Holt
    Sep 23 '20 at 16:01
  • 1
    $\begingroup$ @DerekHolt : Thanks. I agree that it is likely to be true that there is no such example (but I am not totally sure- for almost simple groups it seems very likely, certainly). $\endgroup$ Sep 24 '20 at 8:33
  • 2
    $\begingroup$ I am reminded of the following generalization by Wielandt of a famous theorem of Frobenius. The Theorem of Wielandt asserts that if a finite group $G$ has a subgroup $H$, such that there is a normal subgroup $H_{0} \lhd H$ such that $H \cap H^{g} \leq H_{0}$ for all $g \in G \backslash H$, then there is $K \lhd G$ such that $G = KH$ and $K \cap H = H_{0}$. $\endgroup$ Sep 25 '20 at 20:21

An example was constructed by Pablo Spiga: https://arxiv.org/abs/2102.13614 "A generalization of Sims conjecture for finite primitive groups and two point stabilizers in primitive groups"

  • $\begingroup$ Yes, I have also noticed that paper. Thanks for your comment. We then think this statement holds true for "most" primitive groups. For example, a series of papers by Konygin (references of Pablo Spiga's paper) handle almost all the cases when $G$ is of AS or PA type. $\endgroup$ Mar 3 '21 at 8:56

Sorry, this is not an answer, but rather an application of standard techniques of local analysis to obtain substantial structural information about $H\cap H^g$, in case an example exists. The techniques are inventions of John Thompson, George Glauberman, and Helmut Bender.

Proposition: Let $G$ be a primitive permutation group on a set $\Omega$, and let $\alpha$ and $\beta$ be distinct points in $\Omega$. If $1\ne G_{\alpha\beta}\triangleleft G_\alpha$, then $G_{\alpha\beta}$ has the following properties:

(a) $F^*(G_{\alpha\beta})$ is either a $p$-group for some prime $p$, or the direct product of nonabelian simple groups.

(b) If $G_{\alpha\beta}$ is solvable, then $F^*(G_{\alpha\beta})$ is a $2$-group or a $3$-group.

(c) If $P$ is a Sylow subgroup of $G_{\alpha\beta}$, then no nontrivial characteristic subgroup of $P$ is normal in $G_{\alpha\beta}$.

(d) If $F^*(G_{\alpha\beta})$ is a $p$-group, $p>2$, then $G_{\alpha\beta}$ is not $p$-stable.

(e) (Uses Odd Order Theorem) $G_{\alpha\beta}$ has even order.

Proof: First, part (c). Suppose that $P$ is Sylow in $G_{\alpha\beta}$, $P_0\ne 1$ is a characteristic subgroup of $P$, and $P_0\triangleleft G_{\alpha\beta}$. By the Frattini argument $G_\alpha=G_{\alpha\beta}N_{G_\alpha}(P)\le N_{G_\alpha}(P_0)$ so $G_\alpha=N_G(P_0)$ by maximality of $G_\alpha$. Hence $N_{G_\beta}(P)\le N_{G_\beta}(P_0)=G_{\alpha\beta}$. Since $P$ is Sylow in $G_{\alpha\beta}$, it is Sylow in $G_\beta$ and hence also in $G_\alpha\cong G_\beta$. Now $G_\beta=G_\alpha^g$ for some $g\in G$, so $P^{gh}=P$ for some $h\in G_\beta$, by Sylow's theorem. Then $gh\in N_G(P)\le N_G(P_0)$ so $P_0\triangleleft G_\alpha^{gh}=G_\beta$. Then $P_0\triangleleft\langle G_\alpha,G_\beta\rangle=G$, so $P_0=1$ as $G_\alpha$ contains no nontrivial normal subgroup of $G$. But $P_0\ne1$, contradicttion.

Next, part (a). If $F(G_{\alpha\beta})=1$, then $F^*(G_{\alpha\beta})=E(G_{\alpha\beta})$ and it is the direct product of nonabelian simple groups. So suppose $B=O_p(G_{\alpha\beta})\ne 1$ for some prime $p$. Then $N_G(B)=G_\alpha$. Let $A=O^p(F^*(G_{\alpha\beta}))$, the subgroup of $F^*(G_{\alpha\beta})$ generated by all its $p'$-elements. By the structure of generalized Fitting subgroups, $[A,B]=1$. If $A=1$, then $F^*(G_{\alpha\beta})=B$ and we are done, so assume that $A\ne 1$. Then $N_G(A)=G_\alpha$. Let $a\in A$ be an arbitrary $p'$-element and consider the action of $\langle a\rangle\times B$ on $C=O_p(G_\beta)\triangleleft G_\beta$. We have $[\langle a\rangle,C_C(B)]\le [A,G_\alpha]\cap[G_\beta,C]\le A\cap C\le O_p(A)$. But $A$ is the elementwise-commuting product of $q$-groups for various primes $q\ne p$, and quasisimple groups. Therefore $[\langle a\rangle, C_C(B)]\le Z(A)$ so $[\langle a\rangle,C_C(B),\langle a\rangle]=1$. As $a$ is a $p'$-element normalizing the $p$-group $C_C(B)$, a standard coprime action lemma implies that $[\langle a\rangle,C_C(B)]=1$.Then the Thompson $A\times B$ Lemma implies that $[\langle a\rangle,C]=1$. But $a$ was an arbitrary generator of $A$, so $[A,C]=1$. This implies in turn that $C\le N_G(A)=G_\alpha$. Since $C$ is a normal $p$-subgroup of $G_\beta$, $C\le O_p(G_{\alpha\beta})=B$. But $B\le O_p(G_\alpha)\cong O_p(G_\beta)=C$, so $B=C$. Since $B\triangleleft G_\alpha$ and $C\triangleleft G_\beta$, $B\triangleleft G$, a contradiction as $B\le G_{\alpha}$.

Now (d). If $F^*(G_{\alpha\beta})$ is a $p$-group, then a Sylow $p$-subgroup $P$ of $G_{\alpha\beta}$ is nontrivial. If also $p>2$ and $G_{\alpha\beta}$ is $p$-stable, then by Glauberman's $Z(J)$-Theorem, $Z(J(P))\triangleleft G_{\alpha\beta}$ for any Sylow $p$-subgroup $P$ of $G_{\alpha\beta}$. But $Z(J(P))$ is a nontrivial characteristic subgroup of $P$. This contradicts (c).

Now (b). By (a) and solvability, $F^*(G_{\alpha\beta})$ is a $p$-group for some prime $p$. Suppose $p>2$. By (d), $G_{\alpha\beta}$ is not $p$-stable, whence $G_{\alpha\beta}$ has a subquotient isomorphic to $SL_2(p)$. So $SL_2(p)$ is solvable and $p=3$.

Finally, (e). Suppose that $G_{\alpha\beta}$ has odd order, so it is solvable. By (b) and (d), $F^*(G_{\alpha\beta})$ is a $3$-group and $G_{\alpha\beta}$ is not $3$-stable. Hence $G_{\alpha\beta}$ has an $SL_2(3)$ subquotient, which is of even order, contradiction.

  • $\begingroup$ These are very good criteriums on such $G_{\alpha\beta}$ (of course, if any). Thank you for your observations. $\endgroup$ Sep 27 '20 at 2:25
  • 2
    $\begingroup$ If $G_{\alpha \beta}$ is solvable, I think you can get that either $G$ involves ${\rm Qd}(3)$ or $G$ involves $S_{4}$. In particular, a Hall $\{2,3\}$-subgroup of $G$ is neither $2$-closed nor $2$-nilpotent. This needs a Theorem of Stellmacher, as well as Glauberman's $ZJ$-theorem.( using condition c)). $\endgroup$ Oct 1 '20 at 20:55
  • $\begingroup$ In the above comment, $G$ means $G_{\alpha \beta}.$ $\endgroup$ Oct 3 '20 at 10:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.