Spectral synthesis for central functions on locally compact groups

Question

There is a large literature on harmonic analysis on locally compact group, that I am just beginning to discover. However I have not seen so far anything that emphasizes the central functions on $G$. A good reference on this subject could thus be very useful, as I'd like to understand generalizations to as large as possible a class of locally compact groups of the following elementary results on finite groups $G$:

Any central function $f: G \rightarrow \mathbb C$ can be written in a unique way $$(1) \ \ \ \ \ f = \sum c_\pi \ \ tr\ \pi,$$ where $\pi$ runs in the sum on the set of equivalence classes of complex irreducible representations of $G$. Moreover we have nice formulae relating various norms of $f$ and the $c_\pi$, such as $$(2)\ \ \ \ \ ||f||^2 = \sum |c_\pi|^2 ,$$ and $$(3)\ \ \ \ \ \ sup_{||h||=1, h \in L^2(G)} ||f \ast h|| = sup_{\pi} |c_\pi|,$$ where $||f||=\sqrt{\int |f(g)|^2 \ dg}$, $dg$ being the Haar probability measure, and $f\ast h$ is the usual convolution product.

Let me explain what kind of generalization I am looking for, focussing first on (1). First, how to define the $c_\pi$ for a given $f$? For $G$ a finite group, $$ (4) \ \ \ c_\pi = \int f(g) \overline{tr \ \pi(g)} dg = \overline{ tr\ \pi(\bar f)},$$ where $\pi(f) := \int f(g) \pi(g) \ dg$ as usual. Thanks to the answer and comments to my earlier question, one can generalize this definition to the following situation: $G$ is second countable, unimodular, type I (hence has a Plancherel measure on its dual $\hat{G}$), and $f$ satisfies some regularity condition (edit: specifically, I'd like to take $f$ in the Eymard's Fourier algebra $A(G)$, i.e. the convolution of two functions in $L^2(G)$). Then $\pi(f)$ is trace-class for almost every $\pi$ for the Plancherel measure, and one can define $c_\pi(f)$, almost, everywhere on the support of the Plancherel measure, that is on the reduced dual $\hat{G}_r$) by the same formula (4) as in the case of finite group. But then I cannot answer these questions, that seem natural to me:

If $f$ is moreover central, is it true that the $c_\pi(f)$ for $\pi$ in the reduced dual determine $f$? If so, is there a formula analogue to (1)? If not, what if we assume in addition that the group $G$ is amenable, so that the dual and the reduced dual are the same? What about (2) and (3)?

PS: I somehow feel that as an ex-PhD-student of an expert in the trace formula, I should know the answer to all this inside-out. Fortunately, my ex-advisor does not read MO :-)

Answer 1

This a long comment, which indicates the difficulties and gives a decomposition of measures in terms of orbital integrals instead of irreducible reps.

As you have noticed yourself, there do not exists many continuous functions, which are invariant under conjugation. This was my comment with the closure of conjugacy classes.

What kind of object is $tr\; \pi$ (assuming it exists)?

I give several suggestions for $G$ being the $F$-points of a reductive group ($F$ local field). These are type I, seperable, unimodular.

1. The most common definition yields that it is a distribution on $C_c^\infty(G)$ satisfying $$ tr\; \pi( \phi \ast \theta) = tr\; \pi( \theta \ast \phi)$$
2. Equivalently, it is a distribution on $C_c^\infty(G)$ satisfying $$ tr\; \pi( \phi^g) = tr\; \pi(\phi), \qquad \phi^g(x)= \phi(g^{-1}xg)$$
3. There eists a locally integrable function $\theta_\pi$ on $G$ with $$ tr \pi(\phi) = \int_G \theta_\pi(g) \phi(g)\; dg.$$ Here, $\theta_\pi$ is necessarily conjugation invariant. Some people refer to the trace meaning the function $\theta_\pi$, which is like identifying a distribution and a generalized function.

Suggested conjecture: Every locally integrable central function is a direct integral of $\theta_\pi$'s?

Moreover, $tr\; \pi$ are extremal algebra states iff $\pi$ is irreducible. That means they can not be written in terms of linear combination of other things. On the other hand, there are orbital integrals, which have the somehow the same properties. Note that there are variants of Plancherel theorems in terms of orbital integrals by Harish-Chandra.

A integral decomposition of measures (not distributions though, but functionals on $C_c(G)$) into extremal(=ergodic) measures is known as Chocquet theory, see e.g. Ergodic decomposition of quasi-invariant measure respective Theo Buehler's suggestion: http://matwbn.icm.edu.pl/ksiazki/cm/cm84/cm84217.pdf

Apply this to $G$ acting itsself by conjugation and you have a decomposition of the Haar measure (take the Haar measure multiplied by some non-vanishing function since the article assumes probability measure)

This together with theorem 1 (5) in James Glimm's article http://www.ams.org/journals/tran/1961-101-01/S0002-9947-1961-0136681-X/S0002-9947-1961-0136681-X.pdf applied to $G$ acting on itsself by conjugation yields that the measures are supported on a single orbit under additional hypothesis.

Theorem: Let $G$ be a 2nd countable, locally compact group of with relatively open conjugacy classes, then for every conjugation invariant function $f$ on $G$ the measure $$f(g) \; dg $$ decomposes into a direct integral of measures, which are each supported only on one conjugacy class.

This theorem applies to reductive groups over local field. I am not sure how to implement this to get something with irreducible rep instead of "orbital integrals" though. The unitary dual of a type I group is $T_0$ as requested by Glimm's theorem, but how to move on? Moving from conjugacy classes to irreducible reps can only be done via dualities, as you might know from the Arthur trace formula.