Existence of a pair of matrices in SL(2,Z) satisfying certain constraints on the spectral radius

Question

Some background: my coauthors and I are working on a problem which deals with the exponential growth rates of certain infinite products of matrices. One of the sub-problems which arises in this project is to prove that a particular function from the unit interval to the reals, which describes the growth rates of a family of related infinite matrix products, does not have any line segments in its graph. Now, the existence of a line segment would imply the existence of a pair of matrices some of whose characteristics coincide in a precise way. We have found an analytic proof that the graph cannot contain line segments, but it is long and displeasingly ad-hoc, and we would rather like to replace it with a shorter argument! Hence, we have found ourselves wondering whether there is an algebraic obstacle which would prevent these coincidences from occurring.

The question which arises from this research, then, is:

Do there exist matrices $A,B \in SL(2,\mathbb{Z})$ which satisfy the following constraints: the matrices $A$ and $B$ have only positive entries, do not commute, and satisfy $$\rho(AB)=\rho(A)\rho(B),$$ $$\mathbb{Q}\left(\rho(A)\right)=\mathbb{Q}\left(\rho(B)\right) \neq \mathbb{Q},$$ where $\rho$ denotes spectral radius?

The non-existence of such a pair of matrices would imply the non-existence of the line segments mentioned above.

One of my coauthors suspects that the spectral radius constraints on $A$ and $B$ cannot be met unless $A=C^k$, $B=C^\ell$ for some $C \in SL(2,\mathbb{Z})$ and $k,\ell \in \mathbb{N}$, which would contradict the hypothesis that $A$ and $B$ do not commute. Any solutions which end up being used in our paper will of course be gratefully acknowledged!

Answer 1

This is true: if one has two matrices $A, B \in SL_2 C$, and one knows the three traces $tr(A),tr(B),tr(AB)$, then this uniquely determines the matrices $A,B$ up to conjugacy if $A$ and $B$ generate a non-elementary discrete group (except in a few degenerate cases which won't occur for positive matrices). See for example Maclachlan-Reid. Since $tr(X) = \rho(X)+\rho(X)^{-1}$ for a positive matrix $X$, this implies that the three matrices have the same traces as two commuting matrices with the same traces, and therefore $A,B$ must be conjugate to a pair of commuting matrices. In fact, as your coauthor suspects, then they must be powers of the same matrix.

Addendum: I'll give some details of the argument, and I forgot a condition ($A$ and $B$ might generate a solvable group otherwise, although this is not possible for discrete groups).

Assume that $A, B \in SL_2 Z$. We want to show that $tr(A),tr(B),tr(AB)$ uniquely determine $A$ and $B$ up to conjugacy in $SL_2 C$.

Since $A$ is a positive matrix in $SL_2 Z$, we may conjugate it in $SL_2 R$ to be of the form $$ A=\begin{pmatrix} \lambda & 0 \\\ 0 & \lambda^{-1} \end{pmatrix} $$ where $\lambda >1$ (because $tr(A)>2$). We conjugate $B$ by the same matrix, and relabel so that $$ B=\begin{pmatrix} a & b \\\ c & d \end{pmatrix}. $$

If $A$ and $B$ commute, then we check that $b=c=0$, so in particular $ad=1$. Conversely, if $ad=1$, then we conclude that $bc=0$, and therefore $b=c=0$ since $A,B$ cannot generate a noncommutative solvable group since $SL_2 Z$ is discrete (see below).
From now on, let's assume that $A$ and $B$ don't commute.

The centralizer of $A$ in $SL_2 C$ consist of matrices of the form $$ \begin{pmatrix} z & 0 \\\ 0 & z^{-1} \end{pmatrix}. $$

We may conjugate $B$ by the centralizer of $A$ to be of the form $$ B=\begin{pmatrix} a & 1 \\\ ad-1 & d \end{pmatrix}. $$ Here I'm making use of the fact that $A,B$ are conjugate into $SL_2 Z$ to conclude that they don't generate a solvable group. This follows from the Margulis lemma (or an easy exercise). If $B$ had off-diagonal entries which were zero, then $\langle A, B\rangle$ would be solvable. The point is that then we may conjugate by elements of the centralizer of $A$ in $SL_2 C$ to arrange the upper right entry to be 1.

Thus, we have $$tr(A)=\lambda+\lambda^{-1}, tr(B)=a+d, tr(AB)=\lambda a +\lambda^{-1} d.$$

Now we compute $$a= \frac{\lambda tr(AB)-tr(B)}{\lambda^2-1} , d= \frac{\lambda^{-1} tr(AB)-tr(B)}{\lambda^{-2}-1}.$$

Thus, $tr(A), tr(B), tr(AB)$ canonically determine $A,B$ up to conjugacy (of course $\lambda$ is uniquely determined by $tr(A)$ given $\lambda >1$).

Now, we need to check that $\rho(AB) \neq \rho(A)\rho(B)$, where $\rho(A)=\lambda, \rho(B)=\mu$. If not, $tr(AB)=\lambda\mu +(\lambda\mu)^{-1}$ (again since $AB$ is positive). Plugging into the above equations, we conclude that $a=\mu, d=\mu^{-1}$, a contradiction.

I'm sure this argument can be simplified, but I wanted to carry through the sketch of the argument I gave the first go around.

Also, one may see that any subgroup of commuting matrices in $SL_2 Z$ is either powers of a parabolic element, a hyperbolic element, or a finite-order element.