Matrices generating non-discrete subgroups of SL(2,R)

Jorgensen's inequality $\mid \left(Tr\left(A\right)\right)^2-4\mid+\mid Tr\left[A,B\right]-2\mid\ge 1$ gives a necessary condition for two matrices A,B to generate a discrete subgroup of SL(2,R). Are there examples showing that this inequality is not a sufficient condition? Are there methods to prove that two given elements do NOT generate a discrete subgroup?

Background: A recently published paper http://xm10402.reportworld.co.kr/data/paper/view/2882/P2881838.html claims (in Theorem 4.5. in the appendix) that a group generated by two hyperbolic matrices A,B of positive trace is discrete if and only if their axes intersect and the trace of the commutator [A,B] is smaller than -2. This theorem seems a bit strong to me but it is not obvious to me how to construct counterexamples. In fact the condition on the trace of the commutator automatically guarantees that Jorgensen's inequality holds, so the first check for non-discreteness must fail.

-

For more than you ever dreamed possible on the subject, see Jane Gilman's

MR1290281 (97a:20082) Gilman, Jane(1-RTG2) Two-generator discrete subgroups of PSL(2,R). (English summary) Mem. Amer. Math. Soc. 117 (1995), no. 561, x+204 pp. 20H10 (22E40 30F35)

Rosenberger, Gerhard(D-DORT) All generating pairs of all two-generator Fuchsian groups. Arch. Math. (Basel) 46 (1986), no. 3, 198–204. 20H10

-
Thank you for the hints to the literature. To answer my own question: in the case that the hyperbolic matrices A,B have intersecting axes it turns out that the generated group is always discrete - this is Theorem 3.1.1 in Gilman's book. Moreover, Theorem 14.4.1 of that book also gives precise conditions for discreteness in the case that the axes are disjoint. – ThiKu May 10 '12 at 23:35

The answer is negative. If $G$ is a discrete subgroup of $SL(2, {\mathbb R})$ then it cannot contain elliptic elements of infinite order. Thus, let $A$ be an elliptic of infinite order with fixed trace. In order for Jorgensen's inequality to be satisfied, we want $|tr([A,B])|$ be large. Note that $ABA^{-1}B^{-1}=AC$, where $C$ is also elliptic whose trace is the same as the one of $A$. Consider the character variety $X$ of representations of $F_2$ to $SL(2,R)$, where we send free generators of $F_2$ to $A$ and $C$. Then $X$ is parameterized by traces of $A, C, AC$. Thus, if trace of $AC$ is bounded, then be get a bounded domain in $X$. However, if you take a sequence $B_n=g^n$, where $g$ is a fixed hyperbolic translation, then the distance between the fixed points of $A$ and $C_n= B_n g B^{-n}$ tends to infinity as $n\to \infty$. Therefore, the resulting sequence of elements of $X$ diverges to infinity. Thus, we obtain a sequence of pairs $(A, B_n)$ so that $|tr([A,B_n])|$ diverges to infinity. Hence, for all but finitely many $n$'s Jorgensen's inequality holds but the group generated by $A, B$ is nondiscrete.

On the other hand, the result that you mention does seem plausible in view of Goldman's thesis, see here and here. Goldman proves that representations of surface groups to $SL(2, {\mathbb R})$ with maximal Euler number are Fuchsian. (Namely, the representation you mentioned looks like a representation of the fundamental group of the 1-holed torus with maximal relative Euler number.)

-
Why should these representations have maximal Euler number? – ThiKu May 10 '12 at 1:53

A complete description of pairs of matrices in $SL(2,\mathbb{R})$ that generate discrete subgroups (and much better results) can be found in Klimenko, Elena; Kopteva, Natalia All discrete RP groups whose generators have real traces. Internat. J. Algebra Comput. 15 (2005), no. 3, 577–618.

-
I did not see that paper, but how is it an improvement on the references in my answer? – Igor Rivin May 9 '12 at 16:25
I thought the subject was pretty much closed with Jane's book. – Igor Rivin May 9 '12 at 16:25
They describe subgroups in $SL(2,\mathbb{C})$ with some restrictions. This is much more complicated. – Mark Sapir May 9 '12 at 16:26
Ah, they do the complex case. OK. – Igor Rivin May 9 '12 at 16:27
From Igor Belegradek's review: This paper completes the classification of two-generated discrete subgroups of $PSL_2(\mathbb{C})$ such that the traces of generators and their commutator are real. – Mark Sapir May 9 '12 at 16:41