User marty isaacs - MathOverflow

Answer by Marty Isaacs for Induction from cyclic / Sylow subgroup are there any nice properties ?

2012-08-17T23:01:02Z

Here is a partial answer. (Your assumption that $G$ is simple is irrelevant for these remarks.) Suppose, as in your example, that the Sylow $p$-subgroup $P$ has order $p$ exactly. Then every irreducible character $\chi$ with degree divisible by $p$ appears as a constituent of $\lambda^G$ with multiplicity $\chi(1)/p$ for every irreducible character $\lambda$ of $P$.

There is a deep theory (due to Richard Brauer) which describes in some detail what the irreducible characters are of a group with a Sylow $p$-subgroup $P$ of prime order. To illustrate the power of the theory, I will indicate how to compute the multiplicity of an irreducible character $\chi$ in $(1_P)^G$ in the special case where $P = {\text C}_G(P)$. Write $e = |{\text N}_G(P):P|$, so $e$ divides $p-1$. Then except possibly for $(p-1)/e$ so-called $exceptional$ characters, each irreducible character of $G$ with degree not divisible by $p$ has degree congruent to $\pm1$ mod $p$. If $\chi(1) = kp + \varepsilon$, where $\varepsilon = \pm1$, then the multiplicity of $\chi$ in $(1_P)^G$ is exactly $k + \varepsilon$. The exceptional characters have equal degrees, and this common degree is congruent mod $p$ to $\pm e$. (Note that if $e = p-1$, there really is no exception.) If $\chi$ is exceptional with degree $kp + \varepsilon e$, where again $\varepsilon = \pm1$, then $\chi$ appears with mutiplicity $k$ in the induced character. This theory can also be used to find the multiplicity of $\chi$ in $\lambda^G$ for nonprincipal $\lambda \in \text{Irr}(P)$. For nonexceptional $\chi$, the answer is $k$, but the answer depends on the particular character $\lambda$ for the exceptional characters.

In fact, Brauer's theory has been generalized by Dade and others to the case where $P$ is cyclic, but is not of prime order. This is not the place to go into detail, however. One reference for much (but not all) of this theory is the book by G. Navarro from Cambridge University Press.

Answer by Marty Isaacs for Common supplements to terms of descending central filtrations

2012-07-12T16:42:05Z

What is wanted, I suppose, are examples where each of the subgroups $K_i$ has a proper supplement but where no common proper supplement exists. Such examples occur in the infinite cyclic group $C$. First, note that each nonidentity subgroup K of C has a proper supplement. To see thus, observe that the index $|C:K|$ is finite. If we choose a prime $p$ not dividing $|C:K|$ then the (unique) subgroup $H$ having index $p$ in $C$ is a proper supplement for $K$.

Now for the example. Enumerate the prime numbers $p_1,p_2,p_3,\ldots$ in arbitrary order, and let $K_n$ be the unique subgroup of $C$ having index $p_1p_2\cdots p_n$. Then $K_n \supseteq K_{n+1}$, and we argue that $D = \bigcap K_n$ is trivial. Otherwise, $|C:D|$ is some integer $r$, and yet $|C:K_n|$ exceeds $r$ for sufficiently large $n$.

Finally to show that the only common supplement is the whole group $C$, suppose that $H$ is a proper common supplement, and note that $H > 1$, so $H$ has finite index. Let $q$ be a prime divisor of $|C:H|$, and let $n$ be such that $p_n = q$. Then $q$ divides the index of both $H$ and $K_n$, and thus each of these subgroups is contained in the subgroup of $C$ of index $q$. This contradicts $HK_n = C$.

Answer by Marty Isaacs for Textbook source for finite group properties deducible from character table?

2012-06-23T21:23:30Z

Lots of properties related to solvability can be deduced from the character table of a group, but perhaps it is worth mentioning one property that definitely cannot be so determined: the derived length of a solvable group. Sandro Mattarei constructed such examples, including examples of $p$-groups with identical character tables but different derived lengths. I think that no examples are known where the difference in derived lengths exceeds 1, however.

Answer by Marty Isaacs for Blackbox Theorems

2012-06-23T18:14:43Z

Perhaps the existence of "Tarski Monsters" qualifies as a "blackbox" theorem. The theorem is that for sufficiently large prime numbers $p$, there exist infinite groups $G$ such that every proper nonidentity subgroup has finite order $p$. Such a Tarski monster is clearly 2-generated and has finite exponent, so it provides counterexamples to the Burnside problem. Also it is a simple group of prime exponent and it provides counterexamples for many other attempts to generalize properties of finite groups to groups in general.

Answer by Marty Isaacs for The number of group elements whose squares lie in a given subgroup

2012-06-04T20:36:35Z

Here is an easy character-theoretic proof of the fact that given a subgroup $H$ of a finite group $G$ and a positive integer $k$, the number of elements $y \in G$ such that $y^k \in H$ is divisible by $|H|$. Let $\theta_k$ be the class function on $G$ defined by $\theta_k(x)$ = |{ $y \in G \mid y^k = x$ }|. It is well known that this class function is a generalized character. (In other words, it is a $\Bbb Z$-linear combination of irreducible characters.) The number of interest here is $\sum_{x \in H} \theta_k(x)$, which is equal to $|H|[(\theta_k)_H,1_H]$. This is clearly divisible by $|H|$ since the second factor is an integer because $\theta_k$ is a generalized character.

In fact, the coefficient of an irreducible character $\chi$ in $\theta_k$ is the integer I called $\nu_k(\chi)$ in my character theory book. For $k = 2$, this is the famous Frobenius-Schur indicator, whose value lies in the set {0,-1,1}. For other integers $k$, it is true that $\nu_k(\chi)$ is an integer, but there is no upper bound on its absolute value.

Answer by Marty Isaacs for Cosets and conjugacy classes

2012-05-16T23:52:50Z

Let's consider further the question of when it happens that every right coset of $H$ contains a unique element of the class $C$, or in other words, $C$ is a right transversl of $H$ in $G$. Nick Gill expressed an interest in these questions in the case where $G$ is simple. It appears likely that for nonabelian simple groups, it never happens that a class $C$ is a right transversal for $H$, where $H$ is the centralizer of $h \in C$. At least, I can prove that in the special case where $h$ has prime order. In fact, more is true: if $G$ is simple and $h$ has prime order, then $|C \cap H| > 1$.

Suppose $|C \cap H| = 1$. Then in the conjugation action of $h$ on $C$, there is exactly one fixed point, namely $h$. If the prder of $h$ is a power of a prime $p$, it follows that $|C| \equiv 1$ mod $p$, and thus $|G:H| = |C|$ is not divisible by $p$, and hence $H$ ccontains a Sylow $p$-subgroup $P$ of $G$, and necessarily, $h \in P$. Also, no element of $P$ other than $h$ is conjugate to $h$ in $G$. But if $h$ has prime order, this is impossible in a simple group. If $p = 2$, this follows by Glauberman's Z* theorem, and if $p> 2$, it is a consequence of a result of Artemovich (1988). [Thanks to Nick Gill for telling me about the Artemovich result.]

One could ask how much weaker is the condition $|C \cap H| = 1$ than the original contition, that $C$ is a transversal for $H$ in $G$. Perhaps it is not weaker at all. A few Magma experiments turned up no examples where $|C \cap H| = 1$, but $C$ is not a transversal.

Answer by Marty Isaacs for Non-commutator in simple group?

2012-05-14T20:59:08Z

The proof of the Ore conjecture was recently completed by Liebeck, O'Brien, Shalev and Tiep. The proof depends heavily on the classification of simple groups.

If one weakens the condition that $G$ is nonabelian simple and assumes the much weaker condition that $G' = G$, then lots of examples exist where G contains noncommutators. See, for example, my note in the MAA Monthly 84 (1977) 720-722.

Finally, I mention a character-theoretic condition that an element $x$ of $G$ is a noncommutator. It is that $\sum \chi(x)/\chi(1) = 0$, where the sum runs over all $\chi \in {\rm Irr}(G)$. This sum is always positive if $x$ is a commutator.

Answer by Marty Isaacs for Irreducible mod-p representation of a semidirect product with trivial p-core

2012-03-18T22:05:49Z

Here is an easy non-cohomological proof of the splitting question. The set-up is this. We have a group $\Gamma$ having a minimal normal subgroup $V$, where $V$ is a $p$-group. Also, $\Gamma/V = G$, and $G$ has a normal $p'$-subgroup $H$. In the original question, $H$ was complemented by a cyclic $p$-group in $G$, but we do not need to assume that. Also, in the original question, $V$ was a faithful $G$-module, but we need a much weaker assumption: that $H$ acts nontrivially on $V$. We want to show that $\Gamma$ splits over $V$.

Write $H = K/V$ and by Schur-Zassenhaus, let $X$ be a complement for $V$ in $K$. Let $N = N_\Gamma(X)$. We argue that $N$ is the desired complement for $V$ in $\Gamma$. First, $KN = \Gamma$ by a Frattini argument, using the fact that all complements of $V$ in $K$ are conjugate to $X$ in $K$. It follows that $\Gamma = VN$. Now $V \cap N$ is normal in $G$ since $V$ is abelian. Since $V$ is minimal normal in $\Gamma$, we have either $V \cap N = 1$, as wanted, or $V \subseteq N$. The latter would imply that $X$ centralizes $V$, and this is not the case since $H$ acts nontrivially.

Answer by Marty Isaacs for abelian sylow-p-subgroups

2012-02-20T23:40:39Z

There is also a character-theoretic argument. Suppose $G' \cap Z(G)$ has a subgroup $U$ of order $p$. We want a contradiction. Let $\lambda$ be a nonprinciipal linear character of $U$. Since $U \subseteq P$ and $P$ is abelian, $\lambda$ has an extension to $\mu$, a linear character of $P$. The induced character $\mu^G$ has degree $|G:U|$, which is prime to $p$, so some irreducible constituent $\chi$ of $\mu^G$ has degree not divisible by $p$. Then $\mu$ is a constituent of the restriction $\chi_P$ by Frobenius reciprocity, and thus $\lambda$ is a constituent of $\chi_U$. But $U$ is central, so $\chi_U = \chi(1)\lambda$. Now let $\sigma$ be the linear character det$(\chi)$. Then $\sigma_U = \lambda^{\chi(1)}$, which is nontrivial since $p$ does not divide $\chi(1)$. This is a contradiction, however, since $U \subseteq G' \subseteq {\rm ker}(\sigma)$. [Note that transfer proofs can often be replaced by arguments using the determinant of a character.]

Answer by Marty Isaacs for When does a subgroup H of a group G have a complement in G?

2012-02-07T23:20:54Z

Given $H \subseteq G$, there are a number of conditions sufficient to guarantee that there exists a $normal$ complement for $G$. One of the more interesting of these is due to Frobenius: Assume that $H \cap H^g = 1$ for all elements $g \in G - H$. Then $H$ has a normal complement in $G$. As yet, there is no proof known that does not use characters.

Answer by Marty Isaacs for Representations in characteristic p

2012-02-03T17:43:57Z

Brauer's proof that the number of similarity classes of irreducible representations of $G$ over an algebraically closed field of characteristic $p$ is equal to the number of $p$-regular conjugacy classes of $G$ is ring-theoretic in flavor, and rather tricky. There is also an easy character theoretic proof based on the following ideas. First, the set IBr$(G)$ of irreducible Brauer characters is in bijective correspondence with the irreducible representations, and this set of functions lives in the space $V$ of all complex-valued class functions defined on the set of p-regular elements. Since $\dim(V)$ equals the number of $p$-regular classes, it suffices to show that IBr($G$) is a basis for $V$. The linear independence of IBr$(G)$ is a standard result. To see that IBr$(G)$ spans, use the facts that Irr$(G)$ spans the space of all class functions and that on each $p$-regular class, the value of an ordinary character is a linear combination (and in fact, a sum) of values of Brauer characters.

Answer by Marty Isaacs for Number of n-th roots of elements in a finite group and higher Frobenius-Schur indicators

2012-01-31T18:54:20Z

If $n > 2$, there is NO absolute upper bound on the "higher" F.S. indicator $s_n(\chi)$. This is Problem 4.9 in my character theory book. (A hint is given there.)

Answer by Marty Isaacs for Are there finite metabelian groups with arbitrarily many character degrees?

2012-01-30T22:14:18Z

Much more is true. Let $S$ be an arbitrary finite set of powers of some fixed prime $p$, subject only to the condition that $1 \in S$. Then there exists a class 2 $p$-group (which, of course is metabelian) such that $S$ is exactly the set of degrees of its irreducible characters. This theorem appears in a paper of mine in the AMS Proceedings of 1986 (Volume 96, pages 51--52.)

Answer by Marty Isaacs for A question about the existence of a specific extension of a character.

2012-01-30T22:04:28Z

Given $N \subseteq H \subseteq G$ and an irreducible character $\psi$ of $N$ that has an extension to $H$, it is NOT in general true that every irreducible character $\chi$ of $G$ that lies over $\psi$ must lie over some extension of $\psi$ to $H$. Probably the easiest counterexample is to take $N = 1$ and $H = G$, where $G$ is any nonabelian group. Let $\psi$ be the principal character of $N$. Then of course, $\psi$ extends to $H$, but if $\chi$ is any nonlinear irreducible character of $G$ then $\chi$ does not lie over any extension of $\psi$ to $H$.

A case where it is true that $\chi$ must lie over an extension of $\psi$ to $H$ is where $N$ is normal in $H$ and $H/N$ is abelian. In that case, every character if $H$ lying over $\psi$ is an extension of $\psi$, so $\mu$ can be taken to be an arbitrary irreducible constituent of $\chi_H$.

Answer by Marty Isaacs for Isotropy (aka inertia) of induced representation

2012-01-29T20:21:45Z

The "guess" is wrong; here is a counterexample. Take $G$ to be dihedral of order 16. Let $H$ be one of the two copies of the dihedral group of order 8 in $G$, and let $N$ be one of the two copies of the Klein fours group in $H$. Let $\pi$ be the unique irreducible character of degree $2$ of $H$, and let $\rho$ be one of the two linear constituents of the restriction $\pi_N$.

Now $\rho^H = \pi$ is irreducible, and since $\pi$ is unique, it is invariant in $G$. We argue, however, that $G \ne HT$, where $T = I_G(\rho)$. By definition, $T$ is contained in the normalizer in $G$ of $N$, and of course, $N$ is also normal in $H$. Thus if $G = HT$, it would follow that $N$ is normal in $G$. This is not the case, however. One way to see that $N \not\kern -2pt\triangleleft\ G$ is to observe that otherwise $G/C$ would be embedded in ${\rm Aut}(G)$, where $C$ is the centralizer of $N$. This would force $|C| \ge 8$, and that would imply that $N$ is contained in an abelian subgroup of $G$ of order $8$. The only abelian subgroup of oirder $8$ in $G$, however, is cyclic, so does not contain $N$.

Answer by Marty Isaacs for Finding a subnormal series with specified quotients and end group of specific depth (defect)

2012-01-20T21:16:21Z

This can always be done:

Given nontrivial groups $A_i$ for $0 \le i \le n$, there exists a group $G$ and a subnormal series $H = H_0 < \cdots < H_n = G$ such that $H_i/H_{i-1} \cong A_i$ for $0 \le i < n$ and such that no shorter subnormal series from $H$ to $G$ exists.

Here is my proof:

We can assume $n > 1$, and we induct on $n$. By the inductive hypothesis, let $W$ be a group with subnormal series $V = V_1 < \cdots < V_n$, such that $V_i/V_{i-1} \cong A_i$ for $1 \le i < n$, and such that there exists no shorter subnormal series for $V$ in $W$. Write $A = A_0$ and let $G$ be the wreath product of $A$ with $W$ corresponding to the action of $W$ on the right cosets in $V$. In other words, $G = BW$ is a semidirect product, where $B \triangleleft G$ and $B$ is the direct product of $|W:V|$ copies of $A$. Also, $W$ acts to permute these direct factors of $B$, and this action is permutation isomorphic to the action of $W$ on the cosets of $V$ in $W$. (In fact, we assume that we are given a specific bijection from the set of cosets of $V$ onto the set of direct factors of $B$.)

Now let $C$ be the product of all of the direct factors of $B$ that correspond to nontrivial cosets of $V$, and note that ${\bf N}_W(C) = V$. Let $H = H_0$ be the group $CV$, and for $i > 0$, let $H_i = BV_i$. It is easy to see that $H_0 < H_1 < \cdots < H_n = G$ is a subnormal series with factors $A_i$ as wanted. We must show that no shorter subnormal series for $H$ exists. Note that the subnormal depth of $H_1$ is exactly $n - 1$. (This can be seen by intersecting a subnornal series for $H_1$ in $G$ with $W$. This yields a subnormal series for $V$ in $W$.)

Suppose $H \triangleleft K$. We argue that $BK = BV$. Otherwise, $BK > BV$, so $BK \cap W > V$. But $BK$ normalizes $C$ since $C = B \cap H$, and this contradicts the fact that $V$ is the full normalizer of $C$ in $W$. Now if $H = K_0 < K_1 < \cdots < K_m = G$ is a subnormal series for $H$, then $H_1 = BV = BK_1 \subseteq \cdots \subseteq BK_m = G$ is a subnormal series for $H_1$ with length at most $m-1$, and thus $m \ge n$, as wanted.

Answer by Marty Isaacs for A solvability theorem

2012-01-20T21:50:01Z

Here is an easier proof.

Starting as you did, we get $|G| = 2^a3^b$. Now let $N$ be minimal normal in $G$. Since the hypothesis on maximal subgroups is inherited by $G/N$, it follows (working by induction on $|G|$) that $G/N$ is solvable, so it suffices to show that $N$ is solvable. We can thus assume that $N$ is neither a 2-group or a 3-group. Let $P$ be a Sylow 3-subgroup of $G$, so $S = P \cap N$ is a Sylow $3$-subgroup of $N$, and thus $1 < S < N$, and hence $S$ is not normal in $G$. Let $M$ be a maximal subgroup of $G$ containing ${\bf N}_G(S)$. Since $P \subseteq M$, we see that $|G:M|$ is a power of $2$, so it is $2$ or $4$. By the Frattini argument, $NM = G$, so $|N:N \cap M|$ is $2$ or $4$. Then $N$ has a proper normal subgroup of index dividing $4!$. Then $N' < N$, so $N' = 1$ and $N$ is abelian. This completes the proof.

Answer by Marty Isaacs for characters on a finite group with `extremal' behaviour

2012-01-06T01:44:47Z

All finite groups that are nilpotent with nilpotency class at most 2 are 1-minimal in the sense of this question. Let $\chi \in {\rm Irr}(G)$ and write $Z = {\bf Z}(\chi)$. By Theorem 2.31 of my character theory book, $|G:Z| = \chi(1)^2$ if $G/Z$ is abelian. This condition always holds if $G$ has nilpotence class $2$, because in that case, $G/{\bf Z}(G)$ is abelian, and we certainly have ${\bf Z}(G) \subseteq Z$. By Corollary 2.30, it follows that $\chi$ vanishes on $G - Z$, and we know that this condition is equivalent to saying that $\chi$ is 1-minimal.

Answer by Marty Isaacs for classification of small complete groups

2010-09-30T21:05:36Z

Here's an infinite family of complete groups. Let U be cyclic of prime order p > 2 should and let G be the holomorph of U, so U is normal in G (and in fact is characteristic) and G/U is cyclic of order p-1. It is easy to see that Z(G) = 1. Also U is complemented in G and there are exactly p complements, and they are all conjugate in U. Further, no nonidentity element of G normalizes all of the complements of U in G.

Embed G as a normal subgroup of A = Aut(G). I argue that A = G. Now A permutes the set of complements for U in G via conjugation. If N is the kernel of this action, then N is normal in A. Also, N meet G is trivial by the last sentence of the previous paragraph. It follows that N centralizes G, but since A = Aut(G), we see that N = 1.

Thus A acts faithfully on the set of p complements for U in G, and thus A is isomorphically embedded in the symmetric group S_p. But U is normal in A and U has order p. Since the full normalizer of a cyclic group of order p in S_p has order p(p-1) = |G|, it follows that |A| <= |G| so A = G, and thus G is complete.

It should be mentioned in this context that if G is any finite group with trivial center, then G embeds in Aut(G) and Aut(G) has trivial center, so this process can be repeated, yielding an "automorphism tower" G <= Aut(G) <= Aut(Aut(G)) <= ... . A marvelous theorem of Wielandt shows that this process eventually terminates in a complete group. (For a proof, see my group theory text.)

Comment by Marty Isaacs

2013-04-19T20:39:19Z

Look at the case where the action of G is 3-transitive. Then a point stabilizer is 2-transitive on its (unique) nontrivial orbit. In this case the permutation character of G is the sum of the principal character and one other irreducible, so the (unique) nontrivial irreducible constituent has multiplicity 1. Thus 1 is the best possible lower bound on the multiplicities. That's not very interesting, so I wonder if the first question contains an error, and "upper bound" is intended in place of "lower bound"

Comment by Marty Isaacs

2012-08-28T20:29:53Z

I had not known about Lindsey's theorem that among groups of order n, the cyclic group has the largest possible average (equivalently sum) of element orders. This appeared (as mentioned by Dickman) as a solution to a Monthly problem in Dec. 1991. Independently, and very much later, I found a different proof of this result. (See my paper with Amiri and Amiri in Comm. in Alg. 37 (2009).) Of course, if the Lemma that is the subject of this posting is true, it provides a third proof. It would also prove the corresponding result about the geometric mean. (I have an unpublished proof of that.)

Comment by Marty Isaacs

2012-08-23T22:09:44Z

I think it was Ladisch who mentioned the Hall marriage theorem. In fact, using Hall's theorem, the problem is equivalent to the following: Given a set S of divisors of n = |G|, show that the total number of elements of G having order dividing some member of S is at least as large as for a cyclic group of order n. For example, if S = {a}, then for a cyclic group with order divisible by a the count is a, and by Frobenius' theorem, the count for G is a multiple of a. The inequality thus holds if |S| = 1. I have now found a proof in the case |S| = 2, but I don't (yet) see a general argument.

Comment by Marty Isaacs

2012-08-17T23:15:38Z

There is a purely character theoretic proof that if chi is an irreducible character of G and with defect zero (which means that chi(1) is divisible by the full p-part of |G|), then chi(x) = 0 whenever x has order divisible by p. (This proof depends on Brauer's characterization of characters.) By an amazing result of Knorr, there is a very strong converse. If chi vanishes on every element of order p exactly, then chi has defect zero.

Comment by Marty Isaacs

2012-08-09T18:48:13Z

By Ladisch's comment on the marriage theorem, the existence of a bijection s from G to a cyclic group of the same order such that o(g) divides o(s(g)) for all g in G seems to be equivalent to a kind of generalization of Frobenius theorem: Let S be a set of positive integers. Then the number of elements of G with order dividing some member of S is at least the corresponding number for a cyclic group of order |G|. Note that if |S| = 1, this follows via Frobenius. (And Frobenius actually yields divisibility and not just inequality.) It is not clear, however, if this "theorem" is actually true,

Comment by Marty Isaacs

2012-07-19T19:52:45Z

The proof given in that Monthly solution seems to me to be at best incomplete. I agree (in the notation of that problem) that the sets S_d for divisors d of k do not consume too much of the set G_k, but what about sets S_d where d < k but d is not a divisor. Perhaps they involve some elements of G_k. Does anyone see a way to repair this problem?

Comment by Marty Isaacs

2012-06-09T22:31:33Z

Asking if associativity is stronger than Hall-Witt amounts to asking if Hall-Witt impliee associativity. But what would that mean? To state Hall-Witt, you need commutators and conjugation. How would you define those in a nonassociative context?

Comment by Marty Isaacs

2012-06-05T22:12:31Z

It seems to be easy to prove via character theory that if |W| = 1, then number of homomorphisms f such that f(W) <= H is a multiple of |H|. I don't see a proof along these lines if W has cardinality exceeding 1.

Comment by Marty Isaacs

2012-05-29T19:13:22Z

Jeff, I have never seen that fact about elements of S_n in classes of size not divisible by n; it's amusing. It was not clear from your post whether or not you actually have a proof. Do you?

Comment by Marty Isaacs

2012-05-18T23:56:30Z

I have finally found a counterexample to the question I asked in the last paragraph of my answer, above. The group is of order 168 = 2^3 3 7, constructed as a semidirect product of a nonabelian group of order 21 acting faithfully on an elementary abelian group of order 8.

Comment by Marty Isaacs

2012-05-15T23:40:52Z

John, No I don't know. That's an interesting question.

Comment by Marty Isaacs

2012-02-22T21:52:53Z

In fact, the stabilizer of two points is a permutation group of degree p, so by Burnside, it is either 2-transitive or else solvable of order pm, where m divides p-1. This means that the original group is either 4-transitive or has order of the form (p+2)(p+1)pm. I agree with Derek that to finish off the 4-transitive case probably requires CFSG, but I wouldn't be surprised if the not-4-transitive case can be done without the classification. Note that if m = p-1, the group cannot occur if p > 2 because in that case it would be sharply 4-transitive, and the only such are S_4, A_6 and M_11.

Comment by Marty Isaacs

2012-02-08T17:04:48Z

I wonder if even the weaker condition that all class sizes are squares implies that G is nilpotent. Does anyone know a counterexample to that? A very tiny step in that direction is the observation that if |G| is divisible by some prime p to the first power only, then a Sylow p-subgroup is central. That is because no class size can be divisible by p, and so the centralizers of the Sylow p-subgroups cover G. But in general, conjugates of a proper subgroup of a finite group can never cover the group.

Comment by Marty Isaacs

2012-01-31T19:33:08Z

It is interesting that the set of "invisible" automorphisms forms an abelian subgroup of Aut(G). To be more precise: Let H be any subgroup of G and let A be a group of automorphisms of G fixing each element of H and each right coset of H in G. Then [G,A] is contained in H, so [G,A,A] = 1. Obviously then, [A,G,A] = 1. By the three-subgroups lemma, [A,A,G] = 1. Thus means that the derived subgroup A' = [A,A] acts trivially on G. Since we are working with automorphisms, this says that A' = 1, so A is abelian. There is much more about this sort of thing in Section 4C of my group theory book.

Comment by Marty Isaacs

2012-01-31T19:15:37Z

Actually, more is true. Let H be a proper subgroup of a finite group G. Not only is it true that some element of G lies in no conjugate of H, but in fact, there must be at least |H| such elements. This can be proved by a variation on the argument given in the comment by Harden.