8 minor typos

I revise earlier edits to give a coherent account of the construction which shows that such subgroups can exist.

The underlying idea of the strategy is as follows: Let $X$ be a non-trivial finite group with trivial center which admits an automorphism $\alpha$ fixing only the identity ((informally, and by a slight abuse, known as fixed-point free automorphism). Note that $\alpha$ can't have prime order, for otherwise $X$ would be nilpotent by Thompson's theorem, and hence would have non-trivial center. Then the direct product $X \times X$ is a product of two self normalizing subgroups of order $|X|$ which intersect trivially. One is $\Delta(X) = \{ (x,x): x \in X \}$. The other is $\Delta^{\alpha}(X) = \{ (x,x\alpha): x \in X\}$. That $\Delta(X)$ is self-normalizing is clear, since $Z(X) = 1$. For notice that if $(x,x)^{(a,b)} \in \Delta(X)$ for each $x \in X$, then $ab^{-1} \in Z(X)$. The argument for the other subgroup is similar. Now we seek such a group $X$ with trivial center admitting a fixed point free automorphism $\alpha$ of composite odd order. If such a group exists, we may set $\beta = \alpha^{-1}$. Then the group $G = X \times X$ has the desired three self-normalizing subgroups $\Delta(X)$, $\Delta^{\alpha}(X)$ and $\Delta^{\beta}(X)$. For notice that if $x \alpha = x \beta$, then $\alpha^{2}$ fixes $x$, so that $\alpha$ fixes $x$ as $\alpha$ has odd order. But then $x$ is the identity by hypothesis. Hence any two of the three subgroups have trivial intersection, and have product $X \times X$ by order considerations.

There does exist a group $X$ of order $7^{6} .2^{4}$ which admits a fixed point free automorphism $\alpha$ of order $9$, and which has trivial center. Hence the construction of the construction above does work in this case, and $G = X \times X$ has three self-normalizing subgroups of the required form. We find a subgroup $Y$ of order $144$ of ${\rm GL}(6,7)$ which has an elementary Abelian normal subgroup $U$ of order $16$, acted on by an element $a$ of order $9$ whose cube centralizes $U$ but such that $a$ itself acts fixed-point freely on $U$, and such that, furthermore, $a$ does not have the eigenvalue $1$ in the given representation. We then take $X$ to be the semidirect product $VU$, where $V$ is a $6$-dimensional vector space over ${\rm GF}(7)$ and $U$ acts as the given elementary Abelian subgroup of ${\rm GL}(6,7)$. Then allowing $a$ to act on $U$ as it does within ${\rm GL}(6,7)$, and to act on $V$ as theh the given matrix yields an action of $a$ on $VU$ as a fixed-point free automorphism of order $9$. In terms of matrices, $a$ is the matrix $( [0,1 , 0 ,0 , 0 , 0],[0 ,0, 1 , 0 , 0 , 0], [2 , 0 , 0 , 0, 0, 0], [0,0, 0 , 0 ,1 , 0],[0 , 0 , 0 , 0 , 0 , 1],[ 0 , 0 , 0 , 4, 0, 0])$. The group $U$ is the set of diagonal matrices with diagonal entries $\pm 1$ such that the product of the first three diagonal entries is $1$ and the product of the last three diagonal entries is $1$. The subgroup $U$ is $a$-invariant, $a^3$ centralizes $U$, but $C_{U}(a) = I$. (Could not get latex right for matrix). The group $VU\langle a \rangle$ is the semidirect product $VY$. A similer similar construction works for other odd primes $p$ by considering ${\rm GL}(2p,q)$, where $q$ is a prime congruent to $1$ (mod p).

7 Rewrote to make coherent account.

Here's

I revise earlier edits to give a strategy coherent account of the construction which might merit some thought (later edit- and it works)shows that such subgroups can exist.

The underlying idea of the strategy is as follows: Let $X$ be a non-trivial finite group with trivial center which admits an automorphism $\alpha$ which fixes fixing only the identity (note (informally, and by a slight abuse, known as fixed-point free automorphism). Note that $\alpha$ can't have prime order, for otherwise $X$ would be nilpotent by Thompson's theorem)theorem, and hence would have non-trivial center. Then the direct product $X \times X$ is a product of two self normalizing subgroups of order $|X|$ which intersect trivially. One is $\Delta(X) = \{ (x,x): x \in X \}$. The other is since $Z(X) = 1$. For notice that if $(x,x)^{(a,b)} \in \Delta(X)$ for each $x \in X$, and the then $ab^{-1} \in Z(X)$. The argument for the other subgroup is similar. I look for Now we seek such a group $X$ with trivial center admitting a fixed point free automorphism $\alpha$ of composite odd order. Set If such a group exists, we may set $\beta = \alpha^{-1}$. Then I the group $G = X \times X$ has the desired three self-normalizing subgroups $\Delta(X)$, $\Delta^{\alpha}(X)$ and $\Delta^{\beta}(X)$. Notice For notice that if $x \alpha = x \beta$, then $\alpha^{2}$ fixes $x$, so that $\alpha$ fixes $x$ as $\alpha$ has odd order. But then $x$ is the identity by hypothesis. Hence any two of the three subgroups have trivial intersection, and have product $X \times X$ by order considerations.

(Second edit deleted as now redundant).

Third edit:

There is does exist a group $X$ of order $7^{6} .2^{4}$ which admits a fixed point free automorphism $\alpha$ of order $9$, and which has trivial center. Hence the construction ofthe first edit construction above does work in this case, and $G = X \times X$ has three self-normalizing subgroups of the required form. We find a subgroup $Y$ of order $144$ of ${\rm GL}(6,7)$ which has an elementary Abelian normal subgroup $U$ of order $16$, acted on by an element $a$ of order $9$ whose cube centralizes $U$ but such that $a$ itself acts fixed-point freely on $U$, and such that, furthermore, $a$ does not have the eigenvalue $1$ in the given representation. We then take $X$ to be the semidirect product $VU$, where $V$ is a $6$-dimensional vector space over ${\rm GF}(7)$ and $U$ acts as the given elementary Abelian subgroup of ${\rm GL}(6,7)$. Then allowing $a$ to act on $U$ as it does within ${\rm GL}(6,7)$, and to act on $V$ as theh given matrix yields an action of $a$ on $VU$ as a fixed-point free automorphism of order $9$. In terms of matrices, $a$ is the matrixlatex right for matrix). The group $VU\langle a \rangle$ is the semidirect product $VY$.A similer construction works for other odd primes $p$ by considering ${\rm GL}(2p,q)$, where $q$is a prime congruent to $1$ (mod p).

6 typos

Here's a strategy which might merit some thought (later edit- and it works). Let $X$ be a finite group with trivial center which admits an automorphism $\alpha$ which fixes only the identity (note that $\alpha$ can't have prime order, for otherwise $X$ would be nilpotent by Thompson's theorem). Then the direct product $X \times X$ is a product of two self normalizing subgroups of order $|X|$ which intersect trivially. One is $\Delta(X) = \{ (x,x): x \in X \}$. The other is $\Delta^{\alpha}(X) = \{ (x,x\alpha): x \in X\}$. That $\Delta(X)$ is self-normalizing is clear, since $Z(X) = 1$, and the argument for the other subgroup is similar. I look for such a group $X$ with trivial center admitting a fixed point free automorphism $\alpha$ of composite odd order. Set $\beta = \alpha^{-1}$. Then I the group $G = X \times X$ has the desired three self-normalizing subgroups $\Delta(X)$, $\Delta^{\alpha}(X)$ and $\Delta^{\beta}(X)$. Notice that if $x \alpha = x \beta$, then $\alpha^{2}$ fixes $x$, so that $\alpha$ fixes $x$ as $\alpha$ has odd order. But then $x$ is the identity by hypothesis. Hence any two of the three subgroups have trivial intersection, and have product $X \times X$ by order considerations.

(Second edit deleted as now redundant).

Third edit: There is a group $X$ of order$7^{6} .2^{4}$ which admits a fixed point free automorphism $\alpha$ of order $9$, and which has trivial center. Hence the construction of the first edit does work in this case, and $G = X \times X$ has three self-normalizing subgroups of the required form. We find a subgroup $Y$ of order $144$ of ${\rm GL}(3,7)$ GL}(6,7)$which has an elementary Abelian normal subgroup$U$of order$16$, acted on by an element$a$of order$9$whose cube centralizes$U$but such that$a$itself acts fixed-point freely on$U$, and such that, furthermore,$a$does not have the eigenvalue$1$in the given representation. We then take$X$to be the semidirect product$VU$, where$V$is a$6$-dimensional vector space over${\rm GF}(7)$and$U$acts as the given subgroup of${\rm GL}(6,7)$. Then allowing$a$to act on$U$as it does within${\rm GL}(3,7)$GL}(6,7)$, and to act on $V$ as theh given matrix yields an action of $a$ on $VU$ as a fixed-point free automorphism of order $9$. In terms of matrices, $a$ is the matrix $\left( \begin{array}{clcrcc} 0 & 1 & 0 & ( [0,1 , 0 & ,0 & 0\\0 & , 0 & , 0],[0 ,0, 1 & , 0 & , 0 & 0\\ , 0], [2 & 0 & 0 & , 0 & , 0 & 0\\ , 0& , 0& , 0], [0,0, 0 & , 0 & ,1 & 0\\0 & , 0],[0 , 0 & , 0 & , 0 & , 0 & 1\\ , 1],[ 0 & , 0 & , 0 & , 4& , 0& 0\\ \right)$, 0])$. The group$U$is the set of diagonal matrices with diagonal entries$\pm 1$such that the product of the first three diagonal entries is$1$and the product of the last three diagonal entries is$1$. The subgroup$U$is$a$-invariant,$a^3$centralizes$U$, but $C_{U}(a) = I\$. (Could not get latex right for matrix).

5 corrected matrix (again)
4 Tidying up
3 Corrected typo, expanded text.
2 Simplified example, and expanded explanation.
1