# Decomposing an arbitrary unitary representation of a connected nilpotent Lie group in terms of its irreps

For a locally compact (Hausdorff) abelian group $G$ we have following theorem (see e.g. Folland):

"For every (strongly continuous) unitary representation $(\pi,\mathcal{H_{\pi}})$ of $G$, there exists a unique regular $\mathcal{H}_{\pi}$-projection-valued measure $P$ on $\hat{G}$ such that $\pi$ decomposes as:

$\pi (g)=\int_{\hat{G}}\left\langle g,\chi\right\rangle dP\left(\chi\right)$ for every $g \in G$."

To which extent is this theorem true for nilpotent Lie groups (say, connected and simply connected)? That is, do we have a canonical decomposition of a unitary representation of such a group in terms of its irreducible unireps and some sort of measure on the unitary dual?

The proof of the above theorem has two major ingredients: the identification of the spectrum of $L^1 (G)$ with $\hat{G}$ when $G$ is abelian and the spectral theory of commutative Banach algebras. It is not clear to me whether any of these ingredients has a suitable analogue in the nilpotent case. Furthermore, in this case $\hat{G}$ is not a group or even a Hausdorff space, plus one would have to integrate an operator-valued function which assumes operators acting on different Hilbert spaces as its values. Thus I am not so sure if the standard theory of projection-valued measures can be so easily applied in this case.

-
I'm only a beginner in the relevant theory, but nilpotent Lie groups are Type I, so their unitary duals are better behaved than the general case. There is a kind of operator-valued Fourier transform which applies to these situations. Hopefully someone who knows this stuff better will come along and leave an answer or a reference. –  Yemon Choi Dec 9 '10 at 19:01
(Most of) Theorem 14.10.5 of Wallach, Real Reductive Groups II: Let $G$ be a locally compact separable topological group, and $\pi$ be a unitary representation of $G$. Then there exists a Borel measure $\mu$ on $\hat G$, and a direct integral of representations of $G$, $\int_{\hat G}\pi_s\ d\mu(s)$, such that $\pi$ is unitarily equivalent to $\int_{\hat G}\pi_s\ d\mu(s)$. –  B R Dec 9 '10 at 21:03
This may be a misfire since I am really only familiar with the $p$-adic case, when it is not necessary to work with unitary representations... but are you familiar with Kirillov's orbit method? It allows you to identify the dual space $\widehat{G}$ of a unipotent group with the space of $G$-orbits in $\mathfrak{g}^∗$ (the dual vector space to the associated Lie algebra $\mathfrak{g}$ with the so-called coadjoint $G$-action by conjugation). –  Justin Campbell Dec 9 '10 at 21:14
To add to my comment, your worries in last paragraph also apply to the real reductive group case, where the decomposition is known. So while there are difficulties, I don't think they are the ones you are anticipating. I don't know much about nilpotent groups, though. Have you looked at the representation theory of Heisenberg groups? It seems to be fairly well documented. –  B R Dec 10 '10 at 0:02
BR - This looks along the lines of what I'm looking for, as long as the this decomposition is sufficiently canonical (i.e. the measure and unitary isomorphism are unique in some strong sense), I'll take a look at Wallach's book, so thanks. Justin - if you know of a good reference on the subject, that will be helpful. –  Mark Dec 10 '10 at 11:07

First some bad news : such a decomposition only exists for groups which are said to be of Type I (some notion coming from the theory of von Neumann algebras). There are examples of topological groups which are not of Type I, and have representations which can be decomposed in two different way (even with disjoint support).

Next the good news : luckily, every connected nilpotent Lie group is of Type I. The spectral measure you are looking for indeed does exist, and in some cases (e.g., the regular representation) it is understood explicitly. See the book by Corwin and Greenleaf or the book by Pukanszky. (It is easy to find them on MathSciNet.)

-

Brad Currey has actually obtained a pretty explicit description of the Plancherel measure for general exponential solvable Lie groups which is used to decompose the representation into direct integral of irreducibles. Notice that nilpotent Lie groups are part of this larger class of exponential Lie groups. He uses the technique of jump indexes and orbit method. Here is his paper

http://mathcs.slu.edu/~currey/030401-Currey-1.pdf

Vignon S. Oussa

-
Does Currey's approach for exponential solvable Lie groups yield more information, when specialized to the case of connected nilpotent Lie groups, than the work of Pukanszjy which that paper references? Or is it just something more general''? –  Yemon Choi Mar 8 '11 at 8:19
Yemon, His formula is very precise and very explicit unlike most Plancherel formulas for Nilpotent Lie groups available in the Literature. Another resource if you want would be "Representations of nilpotent Lie groups and their applications" a book written by Corwin and Greenleaf. Vignon S. Oussa –  Vignon Mar 10 '11 at 3:52
Thanks Vignon - I may have a look, since I find what little I've tried to read of Pukanszky's work both difficult and opaque –  Yemon Choi Mar 10 '11 at 20:22