Decomposition of Regular Representation of Non-compact Lie group

Question

Let G be a non-compact Lie group, such as SL(n,R), GL(n,C).

How does the regular representation $L^2(G)$ decompose? Is there an analogue of Peter-Weyl theorem?

Answer 1

Consider the simplest case! By the Fourier transform, $L^2(\mathbb R,+)$ is isomorphic to $L^2 (\mathbb R)$ with the group action

$x \cdot f( \xi) = f(\xi) e^{2 \pi i x \xi}$

A simple example of a subrepresentation consists of all functions that are zero outside the interval $[a,b]$. But these representations are always decomposable - e.g. into $[a,a+b/2] \cup [a+b/2,b]$. Indeed there are no indecomposible subrepresentations, because the functions involved would have to be supported at a point, but there are no $L^2$ functions supported at a given point.

Thus we already get quite different behavior. Though there are many irreducible represntations, we cannot find them in the regular representation. Instead, the regular representation can be decompossed into smaller representations, that get closer and closer to these irreducible representations in some sense, but are never close enough that they themselves are indecomposable.

Thus, in addition to a Peter-Weyl "discrete spectrum" phenomenon, there is also this sort of "continuous spectrum".

Answer 2

Adding some further detail to Will Sawin's answer, in some examples that are clearer than the general case:

For semi-simple real Lie groups $G$ obtained by "restricting scalars" of ("classical") complex semi-simple Lie groups, Gelfand-Naimark in 1947 (I think...?) gave a Plancherel theorem for $L^2(G)$, involving integrals of (the simplest) irreducible unitary repns, namely, the unitary principal series. The "Plancherel measure" was/is a uniform sort of real-line measure on the spectral parameters. They conjectured that these were all the irreducible unitaries of such groups, but it was observed later that there were "unitary degenerate principal series" that occurred as proper subrepns of non-unitary (and not unitarizable) principal series.

After work of Wigner and Bargmann in the late 1940s on some small examples of semi-simple groups not obtained by restriction of scalars on complex groups, in the early 1950s Harish-Chandra started considering the more general semi-simple real case. Although the "holomorphic discrete series" repns of classical groups "of hermitian type" had been implicit in Siegel's and other's early work on modular forms in several complex variables since the late 1930s, it was only in the 1950s that Harish-Chandra made explicit the point that, for such (non-compact) groups there are non-trivial "discrete series" repns, meaning, literally ("discretely") appearing in the decomposition of $L^2(G)$, rather than appearing "continuously".

It took Harish-Chandra many years to treat the other discrete series repns of semi-simple real Lie groups, and, even then, he reasoned indirectly by constructing their characters rather than having models of the repns.

In fact, constructions of models was an on-going process in the 1960s and 1970s, with W. Schmid, R. Langlands, G. Zuckerman, and others adding new ideas of various sorts. The historical notes in A. Knapp's Princeton Press volume on repn theory of semi-simple real Lie groups include more bibliographic pointers.

Answer 3

The answers by Will and Paul, along with various comments, are probably enough to convince anyone that the question asked is exceptionally broad (too broad for a self-contained answer) and involves a great deal of well-organized theory developed in recent generations. The "group algebra" $L^2(G)$ is especially well studied for semisimple Lie groups, where you could spend considerable time studying the collected works of Harish-Chandra (reviewed nicely by V.S. Varadarajan in Bull. London Math. Soc. 17 (1985)). For some useful expository articles and references to related work, see the volume:

The mathematical legacy of Harish-Chandra. A celebration of representation theory and harmonic analysis. Proceedings of the AMS Special Session on Representation Theory and Noncommutative Harmonic Analysis, held in memory of Harish-Chandra on the occasion of the 75th anniversary of his birth, in Baltimore, MD, January 9–10, 1998. Edited by Robert S. Doran and V. S. Varadarajan. Proceedings of Symposia in Pure Mathematics, 68. American Mathematical Society, Providence, RI, 2000