# Invertible matrices satisfying $[x,y,y]=x$

I have been thinking about this question for quite some time but now this question by Denis Serre revived some hope.

Question. Let $x,y$ be invertible matrices (say, over $\mathbb C$) and $[x,y,y]=x$ where $[a,b]=a^{-1}b^{-1}ab$, $[a,b,c]=[[a,b],c]$. Does it follow that some power of $x$ is unipotent?

The motivation is this. Consider the one-relator group $\langle x,y \mid [x,y,y]=x\rangle$. It is hyperbolic (proved by A. Minasyan) and residually finite (that is proved in my paper with A. Borisov). If the answer to the above question is "yes", then that group would be non-linear which would provide an explicit example of non-linear hyperbolic group.

Update 1. Can $x$ in the above be a diagonal matrix and not a root of 1?

Update 2. The group is residually finite, so it has many representations by matrices such that $x, y$ have finite orders (hence their powers are unipotents).

Update 3. The group has presentation as an ascending HNN extension of the free group: $\langle a,b,t \mid a^t=ab, b^t=ba\rangle$. So it is related to the Morse-Thue map. Properties of that map may have something to do with the question. See two quasi-motivations of the question as my comments .

• Are there any one-relator groups known not to be linear? – Łukasz Grabowski Nov 3 '10 at 23:06
• @Lukasz: Yes, there are even non-residually finite ones: $BS(2,3)=\langle x,y \mid y^{-1}x^2y=x^3\rangle$. There are also residually finite 1-related groups which are not linear. Those were constructed in our paper with Cornelia Drutu (in J. Algebra). The point is that this group is hyperbolic. There is an example of a non-linear hyperbolic group due to M. Kapovich (which easily follows from the super-rigidity of certain rank 1 lattices and a Gromov-Olshanskii theorem). But that example has no explicit presentation. This one would be the first explicit example. – user6976 Nov 3 '10 at 23:12
• Here is one of the quasi-reasons why I think the answer is positive. If $G=\langle x,y \mid [x,y,y]=x\rangle$ is linear, then it has a representation over a number field, hence over $\mathbb{Q}$. Therefore the sequence of indexes of subgroups of finite index of $G$ must grow polynomially (take congruence subgroups). This would imply that certain polynomial maps over finite fields have many quasi-fixed points with long orbits (see our paper with Borisov). The latter seems to be impossible. – user6976 Mar 18 '12 at 14:04
• A trivial observation: setting $z := [x,y]$, the condition $[x,y,y]=x$ is equivalent to the assertion that the pair $(x,z)$ is conjugate to $(xz,zx)$ after conjugation by $y$. So the question is equivalent to the question of whether a pair of matrices $(x,z)$ which has the property of being conjugate to $(xz,zx)$ is such that all the eigenvalues of $x$ (or equivalently, $z$, which is necessarily conjugate to $x$) are roots of unity. Unfortunately, I got stuck after this observation: the conjugacy does give a number of trace identities involving various words in z,x, but not enough of them... – Terry Tao Mar 18 '12 at 19:40
• @Terry: Yes, this was another quasi-reason. Consider $G=\langle x,y\mid x^y=x^2\rangle$. Then in every linear representation of $G$, conjugating $x$ by powers of $y^{-1}$ will produce matrices that are closer and closer to 1. So if $x,y$ are matrices $\lim_{n\to\infty} x^{y^{-n}}=1$. This means that $x$ is a unipotent element "of $y$" in Margulis' terminology, hence $x$ is unipotent. Now we have a similar presentation $\langle x,z,y\mid x^y=xz, z^y=zx\rangle$, so the idea was to show that some power of $x$ satisfies the limit property above. – user6976 Mar 18 '12 at 21:02

The answer is "No". Indeed, consider the 1-related group $G=\langle x,y \mid [x,y,y]=x\rangle$ That group has a presentation $\langle a,b,t \mid a^t=ab, b^t=ba\rangle$ (easy to check). Thus it is an ascending HNN extension of the free group. The group $G$ is hyperbolic (proved by Minasyan using the Bestvina-Feighn combination theorem). By a theorem of Hagen and Wise, the group (and almost every other hyperbolic ascending HNN extensions of a free group) acts geometrically on a finite dimensional CAT(0)-cube complex. By a result of Ian Agol then the group $G$ is virtually special and hence linear, i.e., there exists an injective homomorphism $\phi$ from $G$ to a special linear group (over $\mathbb{Z}$). Since $G$ is hyperbolic, it does not contain nilpotent non-Abelian subgroups. Thus the pair of integer matrices $(\phi(a), \phi(b))$ is an example showing that the answer is "no".

• Were some of the results this answer uses proved after the question was asked? – Peter Samuelson Mar 28 '18 at 8:13
• I was able to prove this group hyperbolic computationally without any trouble - less than a minute of CPU-time. – Derek Holt Mar 28 '18 at 9:44
• The theorem you're using is actually a result of Hagen and Wise. It's Corollary 6.20 of this paper: arxiv.org/abs/1311.2084 . – HJRW Mar 28 '18 at 10:34
• @Yes I know, I gathered that form your post! I, on the other hand, proved it entirely with a computer (although my programs actually verify that geodesic bigons in the Cayley graph are uniformly thin, and then the group is hyperbolic by Papasoglu's result). – Derek Holt Mar 28 '18 at 10:48
• @HJRW: This also solves my problem from Kourovka Notebook. I believe Hagen has contacted the editors. – user6976 Sep 4 '18 at 5:51

Here's a quick test which might disprove your hopes very quickly:

Take $n$ to be small: Try $2$ first, and $5$ is probably near the limit of a computer algebra system. Choose $x$ to be a random $n \times n$ diagonal matrix with determinant $1$, for example, $\mathrm{diag}(17, 1/17)$. Write out your relation, leaving all the elements of $y$ as variables. After clearing denominators, you have $n^2$ simultaneuous homogenous equations in $n^2$ variables. (If I haven't made any dumb errors, they have degree $3n$.) Ask your favorite computer algebra system to solve them for you. If any of the roots are not on the hypersurface $\det y=0$, then you have a counterexample!

• I did it for $n=2$, of course. The conjecture is true in that case. For $n=2$ you can use the trace identities. That reduces dimension to 3 (every pair of matrices is determined by three traces, if I remember correctly). It is written in the paper with Drutu which I mentioned above. For other $n$'s I did not check. There are no trace identities and the computation is too large. – user6976 Nov 4 '10 at 1:37
• @David: Just to clarify my previous comment. Every pair of $2\times 2$- invertible matrices of det 1 is determined by the traces $tr(a), tr(b), tr(ab)$ up to conjugacy. There are polynomial identities allowing to compute the trace of the word $tr(w(a,b))$ if you know $tr(a), tr(b), tr(ab)$. Then the relation $[x,y,y]x^{-1}=1$ gives that certain trace is equal to 2, etc. – user6976 Nov 4 '10 at 2:23
• OK, got it. Yeah, trace identities would be the way to do this for $n=2$, and maybe for $n=3$. I think just writing out the equations should win for $n=4$, though I haven't tried it. But my point was just that you should be doing these basic low dimensional checks, and it sounds like you are. – David E Speyer Nov 4 '10 at 2:59
• @David: My favorite CAS (Maple) refuses to deal even with the 3-dim case. What is your favorite CAS that can do it? – user6976 Nov 4 '10 at 20:11
• @David: You don't need to clear denominators, as you can suppose that y is in SL_n. The degree of the polynomials will be n^2+n, though, not 3n. @Mark: Playing around, I found that there is a matrix x in SL_2(C) of order 6 and y of order 8 such that [x,y,y]=x - this is of course not an answer to your question as x^6 is unipotent. Do you have an explanation for this example, though? – Guntram Nov 5 '10 at 18:41