Topology on the Unitary Dual

Question

Suppose I have a locally compact topological group G. The unitary dual of G is the set of equivalence classes of irreducible unitary representations of G. Now, it seems to me that the sensible way of putting a topology on this space is as follows:

1. Fix a hilbert space Hn of cardinality n.

2. Consider the set R(G,Hn), the set of unitary representations of G on Hn. We can give it the topology of uniform convergence on compact sets. Specifically, reps pn approach p if for any compact K in G and v in Hn, pn(g)v -> p(g)v uniformly on K.

3. Now take the subspace I(G,Hn) of irreducible representations, with the subspace topology. Then quotient by unitary equivalence, and give the resulting space the quotient topology.

4. Finally, take a disjoint union over all (countable) n.

I am not sure, however, if this is commonly done. The popular topology on the unitary dual seems to be the Fell topology. Is what I described equivalent? If not, what advantages does the Fell topology have? Also, there is the perspective that the unitary dual is more importantly a measure space than a topological space- is a topological structure significant or important?

Thanks.

Answer 1

Let me partly comment on "what advantages does the Fell topology have"?

One of "advantages" is that it is compatible with Kirillov's orbit method. Let me quote from Boyrchenko&K paper THE ORBIT METHOD FOR PROFINITE GROUPS AND A p-ADIC ANALOGUE OF BROWN’S THEOREM

An important feature of all four situations mentioned above is that both $\hat G$ and $g^∗/G$ are equipped with a natural topology. The topology on the former is the so-called Fell topology (see §3.2). The topology on the latter is the quotient of the standard (compact-open) topology on $g^*$ Moreover, in all four cases the orbit method bijection turns out to be a homeomorphism. This is a nontrivial result which has useful applications. For an interesting application in the p-adic setting we refer the reader to [GK92]. In the setting of real Lie groups this statement was originally conjectured by Kirillov, who also proved that the bijection $g^ ∗/G$ −→ $\hat G$ is continuous. The proof that this bijection is also open is substantially more difficult, and was given by Ian Brown about 10 years later in [Br73].

Answer 2

As you pointed out in a comment, with the topology that you describe the space of $n$-dimensional representations is closed. This means that the unitary dual with that topology would be just the disjoint union of pieces, each corresponding to a dimension $n$. On the contrary, the nice aspect of the Fell topology is that it mixes all representations with all dimensions, finite and infinite.

It makes sense in the Fell topology to say that a sequence of representations of various dimensions converge to the trivial (one-dimensional) representation. In fact, it is a serious question to ask if the trivial representation is isolated in this topology or not: for discrete groups, that's the definition of Property T.

It also makes sense to say that some finite-dimensional representations converge to some infinite-dimensional ones. This happens for instance here in an elaborate context, but there are also various simple examples. Fell's topology is just a weak topology that considers all the representations as a whole.

Answer 3

Convergence in the Fell topology is equivalent to convergence of matrix coefficients. In the finite-dimensional case, this is equivalent as $\rho_n(g) v \rightarrow \rho(g)v$.

Quote from Vogan (http://dedekind.mit.edu/~dav/iso3.pdf)

Suppose then that G is a real reductive Lie group. Write $\tilde{G}$ for the unitary dual of $G$. The Fell topology on $\tilde{G}$ is defined as follows. Suppose $S \subset \tilde{G}$. An irreducible unitary representation $\pi$ belongs to the closure of $S$ if and only if every matrix coefficient (equivalently, a single non-zero matrix coefficient) of $\pi$ is the uniform limit on compact sets of matrix coefficients of elements of $S$.

(Edit due to comment:) In the infinite-dimensional setting, the definition ask then for $\langle w,\rho_n(g) v \rangle \rightarrow \langle w,\rho(g)v \rangle$. Fortunately, the strong and weak operator topology coincide on norm bounded set.

The Fell topology is natural for various reasons, e.g. (quote from wiki):

If G is a locally compact group, the topology on dual space of the group C*-algebra C*(G) of G is called the Fell topology, named after J. M. G. Fell.

Also as Emerton points out, the Plancherel measure is a Radon measure with respect to the Fell topology. Admittely this doesn't hit the full unitary spectrum, but only the tempered one. Moreover for type I groups, it is an almost Hausdorff space.

Also the Fell topology gives you the the correct topology on the Pontryagin dual of a locally compact abelian group. The Fell topology is dicrete for compact groups.

For the example, the character distributions vary also continuously with the Fell topology in a suitable sense.