Are there noncongruence subgroups (of finite index) of the modular group generated only by 2 or 3 elements?

By the modular group I mean either $$SL(2,\mathbb{Z})$$ or $$PSL(2,\mathbb{Z})$$.

Where can I find examples of these?

Another question: is there a good (ideally analytical, but possibly computer-aided) way to determine if a subgroup of the modular group generated by some given matrices has finite index, and possibly allowing to compute the index?

• For your 2nd question, Stallings gives an algorithm which determines whether a set of elements of a finite rank free group generates a finite index subgroup, and computes the index of that subgroup. His algorithm adapts easily, I believe, to finite free products of cyclic groups. In the case of $PSL(2,\mathbb{Z}) \approx \mathbb{Z}/2 * \mathbb{Z}/3 = \langle a \rangle * \langle b \rangle$, you would first convert your input matrices into words in $a,b$, and then carry out the Stallings algorithm. Jan 10, 2013 at 18:26

The answer to your Question 1. is that there are not rank 2 subgroups of $PSL_2(\mathbb{Z})$, but there are rank 3 subgroups. This follows from a result of Wohlfahrt. He shows in Theorem 5 that any non-congruence subgroup $\Gamma < PSL_2(\mathbb{Z})$ has index $\geq 7$. Misha points out in a comment to his answer that a finite coarea fuchsian group of rank $N$ has $Area(\mathbb{H}^2/\Gamma)\leq 2\pi(N-1)$. Therefore in the case $\Gamma$ is non-congruence, $Area(\mathbb{H}^2/\Gamma)=[PSL_2(\mathbb{Z}):\Gamma] Area(\mathbb{H}^2/PSL_2(\mathbb{Z})) \geq 7\pi/3$. Thus, $N=rank(\Gamma) \geq 3$. Misha's claim (in the non-uniform case) follows from an application of the free product decomposition of non-uniform Fuchsian group into cyclic groups, Grushko's theorem, and Gauss-Bonnet.

Wohlfahrt also gives an example of a non-congruence subgroup of index $7$ in $PSL_2(\mathbb{Z})$. On p. 531, he points out that there exists an index $7$ subgroup with two cusps. One determines that the group has genus $0$, and two cone points of orders $2$ and $3$ respectively. This group is therefore rank $3$, isomorphic to $\mathbb{Z}\ast \mathbb{Z}/2\ast\mathbb{Z}/3$. I didn't check, but I think this also lifts to a rank 3 subgroup of $SL_2(\mathbb{Z})$.

• Can't you just lift the three generators arbitrarily, and produce a noncongruence subgroup of rank 3 of index either 7 or 14? Jan 10, 2013 at 0:16
• Yes, good point. I had in mind showing the preimage is rank 3, but it doesn't matter. Jan 10, 2013 at 1:20
1. For your first question: "Most" 2-generated subgroups of the modular group will have infinite covolume, so they are not congruence subgroups. For instance, take the subgroup generated the upper and lower triangular matrices with off-diagonal entries equal to $$2$$.

2. For your second question: There is Jorgensen's algorithm for constructing fundamental domain for a subgroup of $$PSL(2,\mathbb Z)$$ with the given set of generators, see my answer in (un)decidability in matrix groups After you construct such fundamental polygon you can check if it has any edges contained in the circle at infinity of $$\mathbb H^2$$. This is equivalent to having infinite index. Analytically, finite index is equivalent to finiteness of covolume.

• Perhaps oxeimon is interested in noncongruence subgroups of finite index, which would make this a harder problem (but probably still feasible). Jan 8, 2013 at 18:27
• Noam: Perhaps. Then the question becomes more interesting and I do not know examples, but: On general grounds, if $\Gamma$ is an $N$-generated Fuchsian group of finite covolume then $Area(H^2/\Gamma)\le 2\pi(N-1)$, which gives an upper bound on index of $\Gamma$ in $PSL(2,Z)$. Tim Hsu in "Identifying congruence subgroups of the modular group", Proc. Amer. Math. Soc. 124, 1351-1359, gives an algorithm to determine if given $\Gamma$ is a congruence subgroup or not. Putting two things together, somebody can then search for non-congruence subgroups of rank $\le 3$. Jan 8, 2013 at 19:20
• Using graphs of groups, it is easy to enumerate the finitely many conjugacy classes of subgroups of rank $\le 3$. Each one corresponds to a finite connected bi-partite graph; each vertex in the first set is either of valence 2 or of valence 1 labelled with $\mathbb{Z}/2\mathbb{Z}$; and each vertex in the second is either of valence 3 or of valence 1 labelled with $\mathbb{Z}/3\mathbb{Z}$. Congruence subgroups being normal, each such graph whose self-isomorphism group is NOT transitive on edges is not a congruence subgroup, and the vast majority satisfy that. Jan 9, 2013 at 2:10
• @Lee: principal congruence subgroups are normal, but not a general congruence subgroup, which need only contain a principal one as a subgroup. Jan 9, 2013 at 2:49
• By Theorem 5 of this paper, the smallest index of a non-congruence subgroup is 7 (an example is given right before the theorem). By Misha's estimate, there are no rank 2 non-congruence subgroups. One could then check Wohlfahrt's example to see if it is rank 3. projecteuclid.org/… Jan 9, 2013 at 17:36

A. J. Scholl has written a series of articles on modular forms on noncongruence subgroups. In the article

On the Hecke algebra of a noncongruence subgroup, Bull. London Math. Soc. 29 (1997), no. 4, 395–399

he gives two examples of index 7 subgroups of $\mathrm{SL}_2(\mathbf{Z})$ generated by 3 elements :

$$\Gamma_{4,3} = \Bigl\langle \begin{pmatrix} 1 & 4 \\ 0 & 1 \end{pmatrix},\begin{pmatrix} 2 & 1 \\ 1 & 1 \end{pmatrix},\begin{pmatrix} 1 & -1 \\ 2 & -1 \end{pmatrix} \Bigr\rangle$$

$$\Gamma_{5,2} = \Bigl\langle \begin{pmatrix} 1 & 5 \\ 0 & 1 \end{pmatrix},\begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix},\begin{pmatrix} 2 & 3 \\ 1 & 2 \end{pmatrix} \Bigr\rangle$$

The subscripts refer to the widths of the cusps (both groups have exactly two cusps).

• Assuming your "width" means Wohlfart's "amplitude" in my answer, neither one of these is Wohlfart's example, which would be $\Gamma_{6,1}$ in your notation. I checked and can verify that these are the other two index 7 examples isomorphic to $\mathbb{Z} * \mathbb{Z}/3 * \mathbb{Z}/2$ that I allude to in my answer. Jan 10, 2013 at 17:58
• I checked in Wohlfahrt and yes, amplitude is the same as width. Jan 11, 2013 at 10:00

I've compiled my serial comments on Misha's answer with this new answer, not containing any information unavailable in the other answers, but working out Wohlfart's example mentioned by Agol in more explicit detail. In particular there is more than one distinct conjugacy class (up to orientation reversal) of index 7 subgroups isomorphic to $\mathbb{Z} * \mathbb{Z}/3 * \mathbb{Z}/2$ and with two cusps; I have found three such. Extra information Wohlfart gives regarding amplitude of the cusps pins down which one it is.

The modular orbifold, of genus zero with one cusp, one $\mathbb{Z}/3$ cone point, and one $\mathbb{Z}/2$ cone point, deformation retracts to a graph of groups $G$ consisting of the unique geodesic connecting those two cone points. Covering spaces of the modular orbifold up to covering isomorphism (in the category of orbifolds), correspond to coverings of $G$ up to covering isomorphism (in the category of graphs of groups), the covering graph sitting as a deformation retraction of the covering orbifold. Each finite degree covering graph of $G$ is a finite, connected, bipartite graph of groups, where each vertex in one set is either of valence 2 labelled with the trivial group or of valence 1 labelled with $\mathbb{Z}/2$, and each vertex in the other set is either of valence 3 labelled with the trivial group or of valence 1 labelled with $\mathbb{Z}/3$; all edges are labelled with the trivial group. Its fundamental group is a free product of: $e_\infty$ copies of $\mathbb{Z}$, $e_3$ copies of $\mathbb{Z}/3$, and $e_2$ copies of $\mathbb{Z}/2$. The minimal number of generators equals $e_\infty + e_3 + e_2$, and $e_\infty = 2 \cdot$(genus)$+$(number of cusps)$- 1$. Also, Wolhfart gives a formula for index taken from a paper of Peterrson (which Agol alludes to in his answer), and which amounts to $$index = 6 \cdot e_\infty + 4 e_3 + 3 e_2 - 6$$

Wolhfart's noncongruence example has index 7 with two cusps, one of amplitude 1 and one of amplitude 6. The amplitude is equal to half the number of edges of the edge path in the graph that is obtained from a circle around the cusp by projecting that circle to the graph under the deformation retraction. If one adds the restriction that this is the 3 generator example, one gets $e_\infty = e_2 = e_3 = 1$, and the corresponding hyperbolic orbifold is a twice punctured disc (an open annulus) with one $\mathbb{Z}/2$ cone point and one $\mathbb{Z}/3$ one point. If one requires the cusp at the origin to have amplitude 1 and the cusp at infinity to have amplitude 6, this is enough to pin down the graph in $\mathbb{C}-0$ uniquely (up to isotopy and orientation reversal).

First draw a circle around the origin, subdivided at two vertices into two edges. One of those vertices will have valence 2 in the graph, the other will have valence 3. Attach to the latter vertex an arc of two edges on the outside of the circle; the interior vertex of that arc will have valence 2, and its opposite vertex will have valence 3. Attach to that opposite vertex two arcs: one arc has one edge and opposite vertex of valence 1 labelled $\mathbb{Z}/2$; the other has two edges, interior vertex of valence 2, and opposite vertex of valence 2 labelled $\mathbb{Z}/3$.

$$\DeclareMathOperator\PSL{PSL}\DeclareMathOperator\F{\mathbf{F}}$$Let me provide a purely group-theoretic answer.

Let $$H$$ be a subgroup of finite index in $$\mathrm{PSL}_2(\mathbf{Z})\simeq C_2\ast C_3$$. Then, by Kurosh, $$H$$ is isomorphic to $$C_2^{\ast e_2}\ast C_3^{\ast e_3}\ast \mathbf{Z}^{\ast e_0}$$ for some integers $$e_0,e_2,e_3\ge 0$$ (clearly finite, and satisfying $$e_2+e_3+e_0\ge 2$$).

Since these are virtually free and since in a free group the index determines the rank, the triple $$(e_2,e_3,e_0)$$ determines the index $$f(e_2,e_3,e_0)$$ of $$H$$ in $$\mathrm{PSL}_2(\mathbf{Z})$$. Let me explicitly compute $$f$$.

For any groups $$A,B$$, we have $$A\ast B\simeq A\ltimes B^{\ast A}$$ (a kind of non-commutative wreath product). Hence, if $$A'$$ has index 2 in $$A$$, then $$A\ast B$$ has a subgroup of index 2 isomorphic to $$A'\ltimes B^\ast {A'\sqcup A'}\simeq A'\ltimes (B\ast B)^{A'}\simeq A'\ast B\ast B$$.

Given that $$C_2^{e_2}$$ has a subgroup of index 2 isomorphic to $$\mathbf{Z}^{\ast e_2-1}$$ for $$e_2\ge 1$$ (the kernel of the map onto $$C_2$$ that's identity on each factor), we deduce that $$2f(e_2,e_3,e_0)=f(0,2e_3,2e_0+e_2-1)$$ for $$e_2\ge 1$$.

Similarly, given that $$C_3^{e_3}$$, for $$e_3\ge 1$$ has a subgroup of index 3 isomorphic to $$\mathbf{Z}^{\ast 2(e_3-1)}$$, we deduce similarly that $$3f(e_2,e_3,e_0)=f(3e_2,0,3e_0+2(e_3-1))$$ whenever $$e_3\ge 1$$.

Hence $$f(e_2,e_3,e_0)=f(0,2e_3,2e_0+e_2-1)/2=f(0,0,6e_0+3(e_2-1)+2(2e_3-1))/6$$ $$=\frac16f(0,0,6e_0+3e_2+4e_3-5)$$ whenever $$e_2,e_3\ge 1$$; when one or both equals zero this is still valid, by direct verification and using that $$f(0,0,1+k)=f(0,0,1+sk)/s$$ by the values of ranks of subgroups of finite index in free groups.

Next, $$\mathrm{PSL}_2(\mathbf{Z})$$ has index 1 in itself, so $$f(1,1,0)=1$$. We just checked that $$f(1,1,0)=f(0,0,2)/6$$ by constructing a copy of index 6 of $$F_2$$. So $$f(0,0,2)=6$$, and, in turn, $$f(0,0,n)=f(0,0,1+(n-1))=(n-1)f(0,0,2)=6(n-1)$$. To conclude,
$$f(e_2,e_3,e_0)=\frac16f(0,0,6e_0+3e_2+4e_3-5)= 3e_2+4e_3+6(e_0-1).$$

When the generating rank is two, i.e., $$e_0+e_2+e_3=2$$ (excluding $$e_2=2$$ as $$C_2^{\ast 2}$$ can't occur), we get: $$f(1,1,0)=1,\;f(1,0,1)=3;\;f(0,1,1)=4;f(0,2,0)=2;f(0,0,2)=6.$$

Hence subgroups of finite index of generating rank 2 in $$\mathrm{PSL}_2(\mathbf{Z})$$ are precisely those subgroups of index $$1,2,3,4,6$$. In index 1,2 there is a single such subgroup.

(Sequel: this was unfinished)

So it remains to describe subgroups of index 1,2,3,4,6 in $$\PSL_2(\mathbf{Z})$$.

Index 1 ($$\simeq C_2\ast C_3$$): the whole group (= principal congruence modulo 1).

Index 2 ($$\simeq C_3\ast C_3$$): there is a single one. It is congruence modulo 2, since $$\PSL_2(\F_2)$$ is a dihedral group of order 6.

Index 3 ($$\simeq C_2\ast\mathbf{Z}$$ or $$\simeq C_2^{\ast 3}$$): up to conjugacy there are two such subgroups. One is normal, actually $$\simeq C_2^{\ast 3}$$: it is congruence modulo 3, since $$\PSL_2(\F_3)\simeq\mathrm{Alt}_4$$ has a normal subgroup of index 3. The other ones are non-normal and congruence modulo 2, since $$\PSL_2(\F_2)$$ has a non-normal subgroup of index 3. [One representative consists of those matrices that are $$\pm I_2$$ modulo 3. If I'm correct it has a single conjugacy class of elements of order 2, so this group is isomorphic to $$C_2\ast\mathbf{Z}$$.]

Index 4 ($$\simeq C_3\ast\mathbf{Z}$$ or $$\simeq C_2^{\ast 2}\ast C_3$$): up to conjugacy there are two such subgroups, if I'm correct. One is congruence modulo 3, since $$\PSL_2(\F_3)\simeq\mathrm{Alt}_4$$ has a subgroup of index 4 (of order 3). [This one has no element of order 2, hence is isomorphic to $$C_3\ast\mathbf{Z}$$.] One is congruence modulo $$4$$, since $$\PSL_2(\mathbf{Z}/4\mathbf{Z})$$, which has order 48, has a subgroup of index 4 (of order 12), if I checked correctly.

Index 6 ($$\simeq\mathbf{Z}^{\ast 2}$$ or $$\simeq C_3^{\ast 3}$$ or $$\simeq C_2^{\ast 4}$$) is the most interesting case. I'll develop this later as it requires a number of verifications.