What is known about the structure of finite groups admitting an automorphism where all elements have "norm" one?

Question

Let $G$ be a finite group admitting an automorphism $\sigma$ of prime order $p$. Define the norm map $N:G\rightarrow G$ with respect to $\sigma$ by $N(g)= g\sigma(g)\sigma^2(g)\dotsb\sigma^{p-1}(g)$.

Question. If we know all elements of $G$ have norm equal to $e$, what can be said about the structure of the pair $(G,\sigma)$? In particular, must $G$ be nilpotent?

For instance, if $p=2$, then $G$ must be abelian, but this need not hold for higher primes, e.g., an inner automorphism of a nonabelian $p$ group of exponent $p$.

This condition on an automorphism is a natural weakening of being fixed point free, so its not unreasonable to hope that some of the structural features of that situation hold here.

Answer 1

First a note about terminology. It seems in the literature if $\sigma$ is an automorphism of $G$ such that $\sigma^n = 1$ and $g\sigma(g)\sigma^2(g) \cdots \sigma^{n-1}(g) = 1$ for all $g \in G$, we call $\sigma$ a splitting automorphism.

After some searching (experts could probably give more detail), I found that the main result of

Otto H. Kegel, Die Nilpotenz der $H_p$-gruppen, Math. Z. 75 (1961), 373–376

shows that if $G$ is a finite group with a splitting automorphism of prime order, then $G$ must be nilpotent.

The proof by Kegel is based on results of Hughes and Thompson:

D. R. Hughes and J. G. Thompson, The $H_p$-problem and the structure of $H_p$-groups, Pac. J. Math., 9, 1097–1101 (1959).

For a finite group $G$ and prime $p$, let $H_p(G)$ be the subgroup generated by elements of order $\neq p$. Hughes and Thompson show that if $G$ is not a $p$-group, then either $H_p(G) = 1$, $H_p(G) = G$, or $[G:H_p(G)] = p$.

In the case $[G:H_p(G)] = p$, conjugation by any element $g \in G$, $g \not\in H_p(G)$ induces a splitting automorphism of order $p$ on $H_p(G)$.

Conversely, suppose that a group $H$ admits a splitting automorphism $\sigma$ of prime order $p$. Consider the subgroup $G = H \rtimes \langle \sigma \rangle$ of the holomorph of $H$. The definition of a splitting automorphism is precisely that the coset $H\sigma$ consists of elements of order $p$. Then for any integer $k$ coprime to $p$, the coset $H \sigma^k$ also consists of elements of order $p$. This follows from the fact that the map $x \mapsto x^k$ defines a bijection $H\sigma \rightarrow H \sigma^k$ which maps elements of order $p$ to elements of order $p$.

Hence in this case any $g \in G$, $g \not\in H$ has order $p$, so $H_p(G) \leq H$. If $H$ is not a $p$-group, then $H_p(G) \neq 1$ and by the result of Hughes and Thompson we must have $H_p(G) = H$.

So for finite groups which are not $p$-groups, the study of splitting automorphisms of order $p$ comes down to the study of what Hughes and Thompson call "$H_p$-groups", i.e. groups of the form $H_p(G)$, where $G$ is a finite group such that $[G:H_p(G)] = p$. This is seen as a generalization of Frobenius groups, which relate to fixed point free automorphisms of order $p$.

Hughes and Thompson show that a $p$-Sylow of an $H_p$-group has a normal complement which is nilpotent, so in particular $H_p$-groups are solvable. In his paper Kegel continues from there, and shows that $H_p$-groups are nilpotent.

---

I note the following about the relation between "fixed point free" and "splitting automorphism".

An argument from (Gorenstein, Finite groups, Lemma 1.1 in Chapter 10): suppose that $\sigma$ is a fixed point free automorphism of $G$ of order $n$. Then the mapping $x \mapsto x^{-1}\sigma(x)$ is injective, hence also surjective. Writing $g \in G$ as $g = x^{-1}\sigma(x)$, we see that $g \sigma(g) \cdots \sigma^{n-1}(g) = e$.

So $\sigma$ being fixed point free implies that $\sigma$ is a splitting automorphism.

The converse fails. For $p = 2$, consider $G$ abelian of even order and not elementary abelian. Then $\sigma(g) = g^{-1}$ clearly defines a splitting automorphism of order $2$, but $\sigma$ is not fixed point free. For $p > 2$, consider $G = C_p \times C_p$ and the automorphism $\sigma$ corresponding to $A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} \in \operatorname{GL}_2(p)$. Then $A^p = I$ and $I + A + A^2 + \cdots + A^{p-1} = 0$. Thus $\sigma$ is a splitting automorphism of order $p$, but $\sigma$ is not fixed point free.

Answer 2

I think that when $G$ has order prime to $p$, your conditions is actually equivalent to $\sigma$ being fixed point free on $G$ ( this is already an abuse- "fixed point free" means that only the identity is fixed).

One way round is well known: first note that if $\sigma$ is fixed point free on $G$ and $G$ has order prime to $p$, then whenever $\sigma$ fixes (setwise) a conjugacy class, $C$ say, of $G$, then $\sigma$ fixes an element of $C$ ( since the number of elements of $C$ is a divisor of $|G|$, so is prime to $p$).

Next, for any $g \in G$, $\sigma(N(g)) = g^{-1}N(g)g$, so that $\sigma$ fixes the $G$-conjugacy class of $N(g)$, so fixes an element of that conjugacy class, forcing $N(g) = 1.$

On the other hand, I claim that whenever $\sigma$ has order $p$ and $G$ has order prime to $p$, then the norm condition forces $\sigma$ to be fixed-point free.

For otherwise, $\sigma$ fixes an element $x \in G$ of prime order $q \neq p.$ Then clearly $N(x) = x^{p}$, which forces $x^{p} = 1$, and we already have $x^{q} = 1$. For some integers $a,b$ we have $ap + bq = 1$, and then $x = (x^{p})^{a}(x^{q})^{b} = 1$, a contradiction.

More generally, this argument shows that (even if $|G|$ is divisible by $p$), the norm condition (for an automorphism $\sigma$ of order $p$) forces $C_{G}(\sigma)$ to be a $p$-group (of exponent $p$ as noted in Antoine Labelle's comment).

Later edit: Here is another observation. If the finite group $G$ of order $p$ admits an automorphism $\sigma$ of order $p$ such that all elements of $G$ have norm one, then $C_{G}(P)$ is nilpotent whenever $P$ is a Sylow $p$-subgroup of $G$. The conclusion does not depend on the choice of Sylow $p$-subgroup. However, $\sigma$ permutes the Sylow $p$-subgroups of $G$, and the number of these is prime to $p$. Hence there is a $\sigma$-stable Sylow $p$-subgroup $P$ of $G$. Then $N_{G}(P)$ is also $\sigma$-stable. and so is $C_{G}(P).$ Now by transfer and teh Schur-Zassenhaus theorem $C_{G}(P) = Z(P) \times T$ for some subgroup $T$ of order prime to $p$ of $C_{G}(P),$ and $T$ is characteristic in $C_{G}(P)$, so is $\sigma$-invariant.

Now $C_{T}(\sigma) = 1$ by the norm condition and the earlier argument. By the famous theorem of J.G. Thompson (solving the Frobenius conjecture) , $T$ is nilpotent. Hence $C_{G}(P) = Z(P) \times T$ is also nilpotent.