Which unitary (permutation-)representations are isolated?

Let $G$ be a discrete countable group which acts on a countable set $X$. This action defines a standard unitary representation, called the permutation representation, on the Hilbert space $l_2(X)$ of all functions $X\to \mathbb C$ with finite $l_2$ norm. The Hilbert space have an obvious Hilbert basis consisting of elements $\delta_x$ indexed by $x\in X$ ($\delta_x$ is the function which is 1 on $x$ and 0 everywhere else), and $G$ acts on the basis by permutations.

I have a fairly general question.

When is it possible to deform the permutation representation to some non-conjugate representation of $G$ in $l_2(X)$?

Here "non-conjugate" means that the deformation is not trivially realized by an action of the isometry group of $l_2(X)$. More generally, I would like to know if there is a theory of infinitesimal deformations which may produce non-conjugate representations starting from a permutation representation for specific pairs $(G, X)$; I suppose that, since $G$ is not compact, a general answer is unknown, but it might be known for specific cases. Any reference is welcome.

-
Are you using the Fell topology on the unitary dual? – Ian Agol Feb 17 '12 at 2:13
I am looking at representations on a fixed vector space $V=l_2(X)$, so I suppose we fix some natural topology on the set of all maps from $G$ to $U(V)$ (but any information with any topology is fine). – Bruno Martelli Feb 17 '12 at 8:34

There is for sure no general theory. Your question is interesting already for $X=G$, i.e. the left regular representation of $G$. I am aware only of isolated examples where that has been studied, e.g. free groups. For example, there is a paper by Pytlik and Szwarc

http://www.math.uni.wroc.pl/%7Eszwarc/pdf/acta.pdf

where they deform the left regular representation by conjugating it by the resolvent of some (carefully chosen) unbounded operator; this way they get an analytic family of uniformly bounded representations, parametrized by the unit disk, and which are unitary for real values of the parameter. This has been extended to groups acting on trees in a Comptes-Rendus Note of mine: Cocycles d'arbres et représentations uniformément bornées, C.R.Acad.Sci. Paris, 310 (1990), Sér. I, 703-708.

-
Thank you, this is very interesting. I will read the papers. – Bruno Martelli Feb 17 '12 at 15:54

If you use the Fell topology on the unitary dual, then there is a very strong property of Lubotzky-Shalom, Property FD, which states that finite representations are dense in the unitary dual. In particular, the regular representation is approximated by finite representations, which one can regard as a form of "deformation" (although I'm not sure precisely what you have in mind), or at least the regular rep. is not isolated in the unitary dual. Many groups have this property, such as free groups, surface groups, and fibered 3-manifold groups.

-
Thanks for the reference, I'm curios to see what kind of deformation they construct. That's very nice stuff, anyway. – Bruno Martelli Feb 17 '12 at 15:56
If by deformation, you mean a 1-parameter family of representations starting with the regular rep., I don't think property FD gives this. Rather, it shows that any representation is not isolated in the unitary dual. Since the term "isolated" is in your question, I thought this might be one interpretation though. – Ian Agol Feb 17 '12 at 17:38