Can the infinite von Dyck groups be subgroups of $SU(n)$?

Question

I asked this over stackexchange with no answer so here we are,

I know by constructing some particular cases that I can find unitary matrices $X$, $Y$ and $Z$ such that

$X^m = Y^n = Z^p = XYZ = 1$

with

$$ \frac{1}{m} + \frac{1}{n}+\frac{1}{p} < 1 $$

indicating an infinite von Dyck group unless the fact that the matrices are unitary implies some additional, non-trivial relations between $X$, $Y$ and $Z$. Is it possible for infinite von Dyck or triangle groups to be subgroups of $SU(n)$?

Also, can somebody point to me some references on representation theory of infinite discrete groups?

Answer 1

The question was somewhat sloppy so it was unclear if it was about some or all von Dyck groups admitting embeddings in $U(n)$. If the question was some then Victor's answer clearly suffices.

If the question was about all von Dyck groups, then Victor's answer almost gets there but not quite, since only finitely many of these groups are arithmetic (first observed by Takeuchi who actually listed them all, but also follows from far more powerful finiteness theorem of Borel and Prasad: Only finitely many arithmetic groups with covolume $\le const$). For non-arithmetic groups one has to worry about all Galois conjugations yielding noncompact forms of $SO(3)$. For non-arithmetic groups, one has to use a bit more elaborate version of Victor's answer. First, observe that every von Dyck group $\Lambda$ contains a closed surface subgroup $\Gamma$ of finite index. I will consider only the case when the genus is $\ge 2$ since virtually abelian case is much easier. Then, being a closed surface group, $\Gamma$ is isomorphic to a cocompact arithmetic subgroup $\Gamma'$ of $O(2,1)$. Now, $\Gamma'$ admits an embedding $\rho$ to some $U(n)$ as explained by Victor. To embed (unitarily) the original group $\Lambda$, use the representation induced from $\rho: \Gamma\to U(n)$ to $\Lambda$.

A more interesting question is if every finitely generated subgroup of $SL(n, {\mathbb R})$ embeds in some orthogonal group. It seems that $SL(m, {\mathbb Z})$ is a counter-example for that.

Update: a. Indeed, $SL(n, {\mathbb Z})$ does not embed in any $U(m)$. The same applies to all finitely generated groups which contain distorted cyclic subgroups, see Yves' comments. The simplest example will be Baumslag-Solitary groups $BS(1,p)=\langle a, b| aba^{-1}=b^p\rangle$, $p>1$.

b. There is no way to construct sytematically all irreducible finite dimensional unitary representations of von Dyck groups since (starting in certain dimension) there will be continuous families of such representations. (The situation is very much unlike theory of Lie groups.) One way to see this is by observing that the spaces of irreducible unitary representations (modulo conjugation) of surface groups have positive dimension. Using induction (of representations) one can then prove the same for von Dyck groups.

Here are few links where one can read about surface group representations (including unitary ones):

http://www.math.u-psud.fr/~labourie/preprints/pdf/surfaces.pdf

http://arxiv.org/pdf/math.GT/0509114.pdf

http://arxiv.org/abs/0710.5263

The old paper by Andre Weil "Remarks on Cohomology of Groups" http://www.jstor.org/stable/1970495 is particularly relevant for the discussion of von Dyck groups $\Gamma$, since Weil computes dimension of the Zariski tangent space of $Hom(\Gamma, G)$ in the end of the paper, where $G$ is an arbitrary Lie group. (He does much more, of course.)

c. One can show that every hyperbolic von Dyck group $D(p,q,r)$, except for $D(2,6,6)$ and $D(2,4,6)$, admits a homomorphism to $PU(2)$ whose image is dense. In particular, for every von Dyck group $\Gamma$ (with the above exceptions), one gets an irreducible representation to $U(n), n\ge 3$. By working more carefully, one can probably prove the same for the two exceptions.

d. Situation with irreducible representations to $SU(2)$ is more complicated: There will be more exceptions.

Answer 2

Yes, such embeddings of von Dyck groups (and more generally, of discrete isometry groups of hyperbolic space) can be constructed with the standard machinery from algebraic groups and lattices. Here is a sketch of the construction, but for the sake of simplicity, I'll describe the orthogonal case and only describe the case of the quadratic field. The whole construction, with attributions, is explained well in Lubotzky's book (see also Borel-Wallach). You can treat it as a black box that allows you to embed a subgroup of a group of orthogonal matrices with signature $(n,1)$ and entries in a quadratic field $K$ into the real orthogonal group $O(n+1).$

Let $q$ be the diagonal quadratic form $\sum_{i=1}^{n+1} a_i x_i^2,$ where $a_i$ are non-zero elements of a real quadratic field $K$, and $G=O(q)$ be the isometry group of $q$ viewed as an algebraic group. Denote by $\sigma$ the non-trivial Galois automorphism of the field $K$ and assume that $a_i$ is totally positive for $1\leq i\leq n,$ but $a_{n+1}$ is not. The group $G(K)$ of $K$-points of $G$ embeds into $O(n)\times O(n,1).$ In a plain language, $g\in G(K)$ is a matrix with entries in $K$ that preserves the form $q,$ and it is mapped into the pair of real matrices $(g,g^{\sigma}),$ where $g$ preserves a positively definite quadratic form $q$ and $g^{\sigma}$ preserves the quadratic form $q^{\sigma}$ of signature $(n,1).$ More precisely, by the restriction of scalars from $K$ to $\mathbb{Q}$, $G$ embeds into a product of orthogonal groups over $\mathbb{Q}$ with real forms $O(n+1)$ and $O(n,1),$ and this embedding leads to an embedding $f$ of $G(K)$ into the product of real groups $O(n+1)\times O(n,1).$ Moreover, the composition of $f$ with the projection onto the first factor is an embedding $f'$ of $G(K)$ into $O(n+1).$

Although von Dyck groups are not necessarily arithmetic, they are defined over $\bar{\mathbb{Q}},$ and some of them over quadratic fields. Thus you can embed them into $SO(3),$ although, of course, the image is dense.

A standard example is $q=\sum_{i=1}^n x_i^2 +\sqrt{5}x_{n+1}^2,$ with $K=\mathbb{Q}(\sqrt{5}).$ The conjugate form $q^{\sigma}= \sum_{i=1}^n x_i^2 -\sqrt{5}x_{n+1}^2,$ and it's clear that $q$ is positively definite and $q^\sigma$ has signature $(n,1).$ I think, although I haven't checked the details, that the case $n=2$ takes cares of the von Dyck group $(2,5,5).$