# Is there a maximal subgroup of depth 3?

Let's first define what we mean by depth of a subgroup.

Let $$G$$ be a finite group and $$H$$ a subgroup. Let $$(V_i)_{i \in I}$$ and $$(W_j)_{j \in J}$$ be the irreducible complex representations of $$G$$ and $$H$$ (up to isom.). Consider the bipartite graph $$\mathcal{G}$$ whose vertices are these representations, and with $$d_{ij}$$ edges between $$V_i$$ and $$W_j$$ if $$\langle V_i\vert_H,W_j \rangle = d_{ij}$$. Let $$\mathcal{G}_0$$ be the connected component of $$\mathcal{G}$$ containing the trivial representation $$V_0$$ of $$G$$. Note that $$\mathcal{G}_0$$ can be called the principal block of the decomposition matrix, or the principal graph. Note that $$\Vert \mathcal{G}_0 \Vert^2 = |G:H|$$.

Definition: The depth of $$H \subset G$$ is the distance between $$V_0$$ and a farthest vertex in $$\mathcal{G}_0$$.

Alternative definition (after Noah): depth is the maximum number of applications of induction $$\mathrm{Ind}_H^G$$ or restriction $$\mathrm{Res}_H$$ from $$V_0$$ that generate a new irreducible component (by Frobenius reciprocity).

Note that the depth of $$H \subset G$$ is $$2$$ if and only if $$H$$ is a normal subgroup.

The principal graph for $$\{e\} \subset S_3$$, where the starry vertex is $$V_0$$:

The principal graph of $$\langle (1,2)(3,4) \rangle \subset A_4$$ (depth $$3$$):

The principal graph for $$A_4 \subset A_5$$ (depth $$5$$):

If $$H \subset G$$ is a maximal subgroup of depth $$2$$, then it is easy to see that $$|G:H|$$ is a prime number.
Let $$I_n$$ be the set of indices of maximal subgroups of depth $$n$$ in the finite groups. Then $$I_2 = \mathbb{P}$$.
In order to see what $$I_n$$ looks like for $$n>2$$, we computed the beginning of these sets, more precisely, we computed the subsets $$E_n \subset I_n$$ restricted to $$|G:H| \le 100$$, $$|G| < 10^7$$ and $$n \le 7$$. The results are the following (see full computation and code below):

• $$E_2=\{2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, \dots \}$$,
• $$E_3=\emptyset$$,
• $$E_4=\{3, 4, 5, 7, 8, 9, 10, 11, 13, 15, 16, 17, 19, 23, 25, 27, 28, 29, 31, 32, 36, 37, \dots \}$$,
• $$E_5=\{ 5, 6, 8, 9, 10, 11, 12, 14, 15, 17, 18, 20, 21, 24, 26, 28, 30, 32, 33, 35, 36, \dots \}$$,
• $$E_6=\{ 4, 6, 7, 8, 9, 10, 12, 14, 16, 18, 20, 21, 24, 25, 26, 27, 28, 30, 32, 35, 36, 38, \dots\}$$,
• $$E_7=\{11, 13, 25, 31, 36, 40, 45, 49, 57, 64, 81, 100\}$$.

Surprisingly $$E_3=\emptyset$$, which leads to wonder whether $$I_3 = \emptyset$$ also, in other words:

Question: Is there a maximal subgroup of depth $$3$$?

For people interested in subfactor (planar algebra) theory, the question extends as follows:
Bonus question: Is there an irreducible maximal subfactor of depth $$3$$ and integral index?

Computation

gap> DepthListPrimitive(100,10000000);
[ [ [ 1 ] ], [ [ 2 ], 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97 ], [ [ 3 ] ],
[ [ 4 ], 3, 4, 5, 7, 8, 9, 10, 11, 13, 15, 16, 17, 19, 23, 25, 27, 28, 29, 31, 32, 36, 37, 41, 43, 45, 47, 49,
53, 55, 57, 59, 60, 61, 64, 65, 66, 67, 68, 71, 73, 78, 79, 81, 83, 89, 91, 97 ], [ [ 5 ], 5, 6, 8, 9, 10, 11, 12, 14, 15, 17, 18, 20, 21, 24, 26, 28, 30, 32, 33, 35, 36, 38, 42, 44,
48, 50, 54, 55, 56, 60, 62, 65, 66, 68, 72, 74, 78, 80, 82, 84, 90, 98, 100 ],
[ [ 6 ], 4, 6, 7, 8, 9, 10, 12, 14, 16, 18, 20, 21, 24, 25, 26, 27, 28, 30, 32, 35, 36, 38, 42, 44, 45, 48, 49,
50, 52, 54, 56, 60, 62, 63, 64, 65, 68, 72, 74, 77, 80, 81, 82, 84, 90, 98, 100 ], [ [ 7 ], 11, 13, 25, 31, 36, 40, 45, 49, 57, 64, 81, 100 ] ]


Code (the first function is due to Jack Schmidt, see here)

PrincipalGraph:=function(g,h)
local mat,edges;
mat:=MatScalarProducts(Irr(h),RestrictedClassFunctions(Irr(g),h)); #Print(mat);
edges := Filtered( Cartesian([1..Size(mat)],-[1..Size(mat[1])]), ij -> not IsZero(mat[ij[1]][-ij[2]]));
return edges;
end;;

DepthPrimitive:=function(d,r)
local P,dd,c,cc,PP,a,G,H;
G:=PrimitiveGroup(d,r);
H:=Stabilizer(G,1);
dd:=0;
P:=PrincipalGraph(G,H);
c:=[1];
while P<>[] do
PP:=[];
cc:=[];
for a in P do
if a[1] in c then
elif a[2] in c then
else
fi;
od;
c:=cc;
P:=PP;
dd:=dd+1;
od;
return dd;
end;;

DepthListPrimitive:=function(n,M)
local d,dd,R,r,L;
L:=[[[1]],[[2]],[[3]],[[4]],[[5]],[[6]],[[7]]];
for d in [2..n] do
R:=NrPrimitiveGroups(d);
for r in [1..R] do
if Order(PrimitiveGroup(d,r))<M then
dd:=DepthPrimitive(d,r);
if dd<8 then
if not d in L[dd] then
fi;
fi;
fi;
od;
od;
return L;
end;;


As Noah points out, you are looking for some (core-free) maximal subgroup $$H such that $$1_H^G$$ has nonzero inner product with every irreducible character.

Say $$G=L_2(p)$$ with $$p$$ prime and $$p \equiv 1 \bmod 8$$. Then $$G$$ has a maximal subgroup $$H \cong S_4$$. Every element $$h \in H$$ has diagonalizable preimage in $$SL_2(p)$$, and the conjugacy class of $$h$$ in $$G$$ is determined by $$|h|$$. Note $$|h| \in \{1,2,3,4\}$$ and that $$G$$ has a unique conjugacy class of elements of each such order.

Let $$h_2$$, $$h_3$$ and $$h_4$$ represent, respectively, the conjugacy classes of elements of orders $$2,3,4$$. Using the facts above and direct calculation, you should get that for any irreducible character $$\chi$$ of $$G$$,

$$\langle 1_H^G,\chi \rangle=\frac{1}{24}(\chi(1)+9\chi(h_2)+8\chi(h_3)+6\chi(h_4)).$$

Now, assuming $$\chi$$ is not the trivial character, $$\chi(1)$$ is at least $$\frac{p+1}{2}$$. On the other hand, each of $$\chi(h_2)$$, $$\chi(h_3)$$ and $$\chi(h_4)$$ is a sum of at most two roots of unity. Thus, for large enough $$p$$, you get $$\langle 1_H^G,\chi \rangle>0$$ for all irreducible $$\chi$$, and $$H$$ has depth three.

I found the character table for $$L_2(q)$$ at the web page of Jeffrey Adams,

http://www.math.umd.edu/~jda/characters/characters.pdf

You might be able to pull a similar trick in other cases - take some group $$H$$ and a (projective) complex irrep of $$H$$. This should in many cases reduce to an embedding of $$H$$ in some simple groups of fixed Lie type over various fields, some of these embeddings making $$H$$ maximal. For large enough fields, you might get depth three.

• The prime numbers $p \equiv 1 \bmod 8$ are: $17,41,73,89, \dots$. I checked that $S_4 < L_2(p)$ has depth $4$ for $p=17$ and depth $3$ for $p=41,73,89$. So $p \ge 41$ should be large enough. The smallest simple group with a depth $3$ maximal subgroup is $L_2(27) > A_4$. – Sebastien Palcoux Oct 8 '18 at 12:32

First let’s translate this into purely group theoretic language.

The vertex at depth $$0$$ is the trivial $$G$$-rep, the vertex at depth $$1$$ is the trivial $$H$$-rep, the vertices at depth $$0$$ or $$2$$ are the $$G$$-irreps in $$\mathrm{Ind}_H^G 1$$, the vertices at depth $$1$$ or $$3$$ are the $$H$$-irreps in $$\mathrm{Res}_H\mathrm{Ind}_H^G 1$$, the vertices at depth $$0$$, $$2$$, or $$4$$ are the $$G$$-irreps in $$\mathrm{Ind}_H^G \mathrm{Res}_H\mathrm{Ind}_H^G 1$$, etc.

Let's first ask which $$H$$ and $$G$$ irreps appear eventually after successive induction-restriction. These are exactly the reps which are trivial when restricted to $$N =\cap_g gHg^{-1}$$. We can replace $$H \subset G$$ with $$H/N \subset G/N$$ without changing the graph (or the subfactor). So WLOG assume $$N = \{1\}$$ and so $$G$$ is a transitive subgroup of $$S_n$$ with $$n=|G:H|$$.

So what you want is a maximal subgroup $$H \subset G$$ such that $$H$$ is non-trivial (so there are vertices at depth $$3$$), and every $$G$$-irrep occurs in $$\mathrm{Ind}_H^G 1$$ (so there are no vertices at depth $$4$$).

If the index $$|G:H|$$ is a prime $$p$$. Take $$P$$ a Sylow $$p$$-subgroup of $$G$$. This is just a choice of $$p$$-cycle in $$G$$. The double coset space $$P\backslash G/ H$$ is trivial since $$P$$ is a transitive subgroup of $$S_p$$. So $$\mathrm{Res}_H \mathrm{Ind}_P^G 1 = \mathrm{Ind}_1^H 1$$ only has one copy of the trivial. Hence any nontrivial irrep $$W$$ in $$\mathrm{Ind}_P^G 1$$ when restricted to $$H$$ has no trivial subrep. Hence $$W$$ doesn’t appear in $$\mathrm{Ind}_H^G 1$$.

Then $$\mathrm{Ind}_P^G 1$$ is a trivial representation of $$G$$, and so $$P=G$$ is a $$p$$-group. But $$G \subset S_p$$ and $$p$$ only divides $$p!$$ once, hence $$G$$ is cyclic of order $$p$$ and $$H$$ is trivial. It follows that $$H \subset G$$ is depth $$2$$, contradiction with depth $$3$$.

Conclusion: There is no maximal subgroup of depth $$3$$ and prime index, i.e. $$I_3 \cap \mathbb{P} = \emptyset$$.

• No idea what happens for composite indices. It’s probably very complicated. – Noah Snyder Oct 6 '18 at 15:42

The investigation of the finite simple groups $$G$$ with $$|G|<10^6$$ (with the code below) reveals the following maximal subgroups of depth $$3$$:

• $$A_4 \subset L_2(q)$$ for $$q=27,37,43,53,67,83,107$$,
• $$S_4 \subset L_2(q)$$ for $$q= 41,71,73,79,89,97,103,113$$,
• $$F_7 \subset J_1$$, with $$F_7$$ a Frobenius group, and $$J_1$$ a Janko group,
• $$A_5 \subset L_2(q)$$, for $$q=101,109,125$$.

The first example $$(A_4 \subset L_2(27))$$ has index $$819$$. The matrix of its (bipartite) principal graph is:
$$\left( \begin{matrix} 1 & 1 & 1 & 2 & 2 & 2 & 1 & 1 & 1 & 2 & 3 & 3 & 3 & 3 & 3 & 3 \\ 0 & 3 & 0 & 3 & 3 & 3 & 2 & 2 & 2 & 2 & 2 & 2 & 2 & 2 & 2 & 2 \\ 0 & 0 & 3 & 3 & 3 & 3 & 2 & 2 & 2 & 2 & 2 & 2 & 2 & 2 & 2 & 2 \\ 0 & 3 & 3 & 6 & 6 & 6 & 7 & 7 & 7 & 7 & 7 & 7 & 7 & 7 & 7 & 7 \end{matrix} \right)$$

Then, the following set of indices of these subgroups is a subset of $$I_3$$: $$\{ 819, 1435, 2109, 3311, 4180, 6201, 7455, 8103, 8585, 10270, 10791, 12529, 14685, 16275, 19012, 22763, 23821, 30058, 51039 \}$$

Question: Is $$819$$ the smallest index for a maximal subgroup of depth $$3$$?

Code

PrincipalGraph:=function(g,h)
local mat,edges;
mat:=MatScalarProducts(Irr(h),RestrictedClassFunctions(Irr(g),h)); #Print(mat);
edges := Filtered( Cartesian([1..Size(mat)],-[1..Size(mat[1])]), ij -> not IsZero(mat[ij[1]][-ij[2]]));
return edges;
end;;

DepthSubgroup:=function(G,H)
local P,dd,c,cc,PP,a;
dd:=0;
P:=PrincipalGraph(G,H);
c:=[1];
while P<>[] do
PP:=[];
cc:=[];
for a in P do
if a[1] in c then
elif a[2] in c then
else
fi;
od;
c:=cc;
P:=PP;
dd:=dd+1;
od;
return dd;
end;;

Depth3MaxSubSimple:=function(n,m)
local it,i,G,T,M,x,H,l;
it:=SimpleGroupsIterator(n,m);
l:=[];
for i in it do