Discreteness of a group of hyperbolic isometries

Question

Referring to A question about hyperbolic double torus, there is non-discrete $\Gamma= \left< a,b,c,d~~|~~[a,b][c,d] \right> \subset PSL_{2}(\mathbb{R})$ where $a,b,c,d$ are hyperbolic elements. I wondered what the sufficient and necessary conditions of the representations of $a,b,c,d$ for the discreteness of $\Gamma$. This may be related to some concepts of Teichmuller space...

Answer 1

There are a number of conditions for discreteness and faithfulness (injectivity) of a representation $\Gamma \to PSL_2(\mathbb{R}) = Isom^+(\mathbb{H}^2)$, where $\Gamma$ is the fundamental group of a closed, oriented surface of genus $g \ge 2$.

One is Goldman's theorem which says that a homomorphism $\Gamma \to PSL_2(\mathbb{R})$ is discrete and faithful if and only if the Euler number of the representation equals $\pm (2g-2)$. To define this Euler number, take the induced action of $\Gamma$ on the compactified $\mathbb{H}^2$ to define a closed disc bundle over the surface $S$, and then take the Euler number of this disc bundle. This result is contained in Goldman's thesis, but I do not know a good published reference.

Another condition is that $\Gamma$ is discrete if and only if each 2-generator subgroup is discrete, and then one can use the work of Gilman/Maskit/et. al. to give an algorithmic characterization of when a 2-generator subgroup of $PSL(2,\mathbb{R})$ is discrete. Gilman's paper (referred to in my comment) is a good reference for this material on 2-generator subgroups of $PSL(2,\mathbb{R})$.

ADDED: Another condition is given in work of Feng Luo, "Geodesic length functions and Teichmüller spaces": a representation $\Gamma \to PSL(2,\mathbb{R})$ is faithful and discrete if and only if the restriction of its character to the fundamental group of each essential 3-holed subsphere and 1-holed subtorus of $S$ is the character of a faithful, discrete representation. Furthermore, for each of those restrictions, discreteness is characterized in terms of polynomial equations in the traces.

ADDED: As long as I'm at this, I might as well go the whole hog. There is a whole series of criteria for discreteness and faithfulness of a representation $\Gamma \to PSL(2,\mathbb{R})$ which goes back to a result of Nielsen, which can be found in his 1940 paper "Uber Gruppen linearer Transformationen", and which continues in results of Siegel, Jorgensen, and others. The original paper of Nielsen can be found, in German, in Volume 2 of "Jacob Nielsen: Collected Mathematical Papers". Also in this volume, on page 431, is a short essay by Fenchel on the history of this theorem and its successors. Nielsen's criterion is that the image of every nontrivial element is a translation. Jorgensen's criterion, which Fenchel says is the most general of this series, is that no nontrivial element has image equal to an infinite order rotation.