Integer matrix that does not belong to a free group of rank 2

Question

I'm given two matrices in $SL_2(\mathbb{Z})$

$$ A = \left(\begin{array}{cc} 2 & 3\\ 3 & 5 \end{array}\right), \ \ B = \left(\begin{array}{cc} 5 & 3\\ 3 & 2 \end{array}\right). $$

Then the group $\langle A, B \rangle$ is free of rank 2. Now my problem is to prove that $\langle A, B \rangle$ does not contain any matrix of the form

$$ \left(\begin{array}{cc} 1 & n\\ 0 & 1 \end{array}\right) $$

with $n\in\mathbb Z$ nonzero. That is, no such matrix can be obtained from products of $A$ and $B$ and their inverses.

I started by writing $$ \left(\begin{array}{cc} 0 & 1\\ - 1 & 0 \end{array}\right) \longrightarrow x, \left(\begin{array}{cc} 0 & 1\\ - 1 & 1 \end{array}\right) \longrightarrow y, A \longrightarrow (y x y^{- 1} x^{-1})^2, B \longrightarrow (y^2 x y^2 x^{- 1})^2, $$ but then did not get any further.

Any help is highly appreciated.

Answer 1

This can be proved using the modular diagram in the upper half plane, and the dual tree $T$ of the modular diagram (see below for a discussion of the picture in the link). The modular diagram has vertices at the rational numbers together with $\infty = \frac{1}{0}$. The hyperbolic line $\overline{\frac{p}{r}, \frac{q}{s}}$ is an edge if and only if $ps-qr=\pm 1$ (all rational numbers are in lowest form).

The dual tree $T$ is bipartite, its $0$-cells of valence 2 vertices being where it crosses the edges of the modular diagram, and its $0$-cells of valence 3 vertices being the barycenters of the ideal triangles of the modular diagram. The $0$-cell of valence 2 on the side $\overline{\frac{1}{0},\frac{0}{1}}$ is $$v_0 = 0 + 1i $$ and $v_0$ is connected by a 1-cell, denoted $E_+$, to a 0-cell of valence 3 vertex at $$v_1 = \frac{1}{2} + \frac{\sqrt{3}}{2} i $$ Let $E_-$ denote the reflection of $E_+$ across the $y$-axis, joining $v_0$ to the point $-\frac{1}{2} + \frac{\sqrt{3}}{2} i$.

Consider paths in $T$ with initial edge $E_+$ that terminate on a $0$-cell of valence 2 (that $0$-cell having positive real part). These paths can be identified with words in the letters $L$ and $R$. For example, the path $RLRL$ starts at $v_0$, goes along $E_+$, then turns Right at the valence 3 vertex $v_1$, then continues past the next valence 2 vertex located at $\frac{1}{2} + \frac{1}{2} i$ to turn Left at the next valence 3 vertex, then Right, then Left, and then stops at the next valence 2 vertex.

What to show is the following list of properties of the fractional linear action of $GL(2,\mathbb{Z})$ restricted to $T$:

1. $\pmatrix{2 & 3 \\ 3 & 5}$ has an axis with fundamental domain $RLRL$

2. $\pmatrix{5 & 3 \\ 3 & 2}$ has an axis with fundamental domain $LRLR$

3. $\pmatrix{1 & n \\ 0 & 1}$ has an axis with fundmantal domain $\underbrace{LLLL…L}_{\text{$n$ times}}$. That axis is the unique line in $T$ passing through all the points $k + i$, $k \in \mathbb{Z}$.

By constructing the entire axes of these elements (just continue forward and backward in $T$ with further copies of the fundamental domain), one sees that the intersection of the axis of $\pmatrix{2 & 3 \\ 3 & 5}$ and the axis of $\pmatrix{5 & 3 \\ 3 & 2}$ is therefore equal to $E_- \cup E_+$.

From this, it follows easily that the entire minimal subtree of the free group $\langle A,B\rangle$ intersects the axis of $\pmatrix{1 & n \\ 0 & 1}$ solely in a finite segment, namely the union of the segment $E_- \cup E_+$ connecting $-\frac{1}{2} + \frac{\sqrt{3}}{2} i$ to $v_1 = \frac{1}{2} + \frac{\sqrt{3}}{2} i$ together with the translate of $E_- \cup E_+$ one unit to the right, connecting $v_1$ to $\frac{3}{2} + \frac{\sqrt{3}}{2} i$, and the translate of $E_- \cup E_+$ one unit to the left, connecting $-\frac{3}{2} + \frac{\sqrt{3}}{2}$ to $-\frac{1}{2} + \frac{\sqrt{3}}{2}$.

From this it follows that $\pmatrix{1 & n \\ 0 & 1} \not\in \langle A,B\rangle$ for all $n \ne 0$, because those elements all have the same axis in the minimal tree of $\langle A,B \rangle$ as described above, and that axis would have to be contained in the minimal tree of $\langle A,B \rangle$.

What remains is to show 1 and 2 above (3 being obvious). The general statement is as follows. Denote $$L = \pmatrix{1 & 1 \\ 0 & 1} \quad R = \pmatrix{1 & 0 \\ 1 & 1} $$ Every matrix $\pmatrix{p & q \\ r & s} \in SL(2,\mathbb{Z})$ with $p,q,r,s > 0$ can be factored uniquely as a product of $L$ and $R$; simply use column reduction. The word you get from that factorization, when interpreted as a path in $T$ starting with $E_0$, is a fundamental domain for the axis of $\pmatrix{p & q \\ r & s}$.

So what's left is a column reduction calculation, to verify that $$\pmatrix{2 & 3 \\ 3 & 5} = RLRL, \qquad \pmatrix{5 & 3 \\ 3 & 2} = LRLR $$

Added bonus: Apropos of some of the comments, one can show, by examining the minimal subtree of $\langle A,B \rangle$ constructed above, that every ray in the minimal subtree oscillates infinitely often between $L$ and $R$. Therefore, this group has no cusps, because each ray in $T$ that limits on a cusp is eventually $RRRRRRR…$ or $LLLLLLL….$.

Added: I finally found a good picture on the web, linked to in the first line of the answer. The hyperbolic lines of the modular diagram are in blue, the dual tree $T$ is in red, and one can label the cusps along the bottom of the picture as $-1,0,1,2,3$.

Answer 2

Look at $A, B$ modulo $3.$ They are both equal to $-I,$ so they group they generate modulo 3 contains only $\pm I,$ so if $n \not\equiv 0 \mod 3$ you are done.

Answer 3

With GAP, you can improve Igor Rivin's answer to rule out all $n$ which are not divisible by $252$:

gap> A := [[2,3],[3,5]];; B := [[5,3],[3,2]];;
gap> G := Group(A*One(Integers mod 252),B*One(Integers mod 252));;
gap> Perform(Filtered(AsList(G),
>                     M -> IsOne(M[1][1]) and IsOne(M[2][2])
>                          and IsZero(M[2][1])),
>            Display);
matrix over Integers mod 252:
[ [  1,  0 ],
  [  0,  1 ] ]