The Jacobson radical $\mathfrak{J}(G)$ of a group $G$ is defined by Reinhold Baer in [2] as the intersection of the maximal normal subgroups of $G$ (as noted by Baer the identity $G = \mathfrak{J}(G)$ holds if and only if $G$ has no maximal normal subgroup). His seminal article [2] was written in German and doesn't seem to be widely known. It contains plenty of interesting elementary results, some of which have been reproved multiple times by different authors. Here is a non-comprehensive list.
- $\mathfrak{J}(G/\mathfrak{J}(G)) = 1$.
- $\mathfrak{J}(G)$ is the set of all the elements which can be omitted from any set which generates $G$ as a normal subgroup, i.e., $\mathfrak{J}(G)$ is the set of elements $g \in G$ such for every $X \subset G$, $\langle X, g \rangle^G = G$ implies $\langle X \rangle^G = G$ [2, Satz 3.1.d].
- For any two groups $A$ and $B$, we have $\mathfrak{J}(A \times B) = \mathfrak{J}(A) \times \mathfrak{J}(B)$ [2, Folgerung 3.4].
- If $G$ is soluble then $[G, G] \subset \mathfrak{J}(G)$ [2, Folgerung 2.10].
- Assume that $\mathfrak{J}(G) \neq G$ and that the set of normal subgroups of $G/\mathfrak{J}(G)$ which are contained in $[G/\mathfrak{J}(G), G/\mathfrak{J}(G)]$ satisfies the minimal condition. Then a subset generates $G$ as a normal subgroup if and only if its image generates $G/[G, G]$ [2, Satz 6.4], see [5] for a generalization.
Of course, $\mathfrak{J}(G)$ is a characteristic subgroup of $G$ and it is not difficult to show that $\mathfrak{J}(G)$ contains the Frattini subgroup of $G$, provided that every simple quotient of $G$ has a maximal subgroup (see Corrigendum below).
I discovered the definition of the Jacobson/Baer radical $\mathfrak{J}(G)$ and some of its elementary properties when studying two generalizations of the Andrews-Curtis conjecture. The authors of [7, 8] refer to $\mathfrak{J}(G)$ as the $N$-Frattini subgroup of $G$ and denote it by $W(G)$. The letter $W$ primary alludes to the word "weight", i.e., the minimal number $w(G)$ of normal generators as commonly defined by knot theoricists, see this MO post for more about $w(G)$. But it could also be a(n) (un)conscious tribute to James Wiegold.
In [5], the concept of Jacobson radical is further generalized to $\Omega$-groups for which the first two items above still hold, as well as some other properties [7, Lemma 4.1]. To echo Geoff Robinson's comment, it is proved in [7, Lemma 4.5] that a finite group $G$ is perfect if and only if $G/\mathfrak{J}(G)$ is the direct product of simple non-Abelian groups. The use of $\mathfrak{J}(G)$ is instrumental in the proof of the finitary Andrews-Curtis conjecture [7] and its quantitative enhancements [8].
In a preprint of mine, I made this humble observation about finitely generated groups:
Any two $n$-tuples which generate $G$ as a normal subgroup can be related by a finite sequence of Andrews-Curtis moves if the two following hold.
- $n > w(G)$;
- $G/\mathfrak{J}(G)$ is Abelian, or finite, or a direct product of finitely many non-Abelian simple groups.
Corrigendum:
Let $G$ be any group and let $M$ be a maximal normal subgroup of $G$.
Lemma 4 of [1]
(MR0122885) asserts that if $M$ is contained in some maximal subgroup of $G$, then $M$ contains the Frattini subgroup $\Phi(G)$ of $G$. As a result, $\Phi(G) \subset \mathfrak{J}(G)$ holds if every simple quotient of $G$ has a maximal subgroup, e.g., $G$ is finitely generated or $G$ is soluble.
The existence or non-existence of a simple group without maximal subgroups was an unsolved problem in 1970 when B. H. Neumann reviewed [1]. It was still an open problem in 1974 as the analogous problem for near simple group was unsolved at that time, see [3] (MR0347986).
The existence of an uncountable simple group without maximal subgroups was eventually established by Shelah in 1980 [4]. The existence of a countable simple group without maximal subgroups was established by Ol′shanskiĭ in 1991 [6, Theorem 35.3]. If $G$ is any of these two infinite simple groups, then we have $\Phi(G) = G$ and $\mathfrak{J}(G) = 1$ so that $\Phi(G) \not\subset \mathfrak{J}(G)$.
[1] V. Dlab and V. Kořínek, "The Frattini subgroup of a direct product of groups", 1960.
[2] R. Baer, "Der reduzierte Rang einer Gruppe", 1964.
[3] J. Riles, "The near Frattini subgroups of direct products of groups", 1974.
[4] S. Shelah, "On a problem of Kurosh, Jónsson groups, and applications", 1980.
[5] A. Rhemtulla, "Groups of finite weight", 1981.
[6] A. Ol′shanskiĭ, "Geometry of defining relations in groups", 1991.
[7] A. Borovik, A. Lubotzky, and A. Myasnikov, "The finitary Andrews-Curtis conjecture", 2005.
[8] R. Burns and D. Oancea, "Recalcitrance in groups II", 2012.