Trace Class Functions on locally compact groups

Let $G$ be a locally compact subgroup, $\mu$ a Haar-measure. For $f \in L^1(G)$, and for $\pi$ a unitary, topology irreducible, representation of $G$ on an Hilbert space $H_\pi$, it is customary to define a continuous operator $\pi(f)$ on $H_\pi$ by $$\pi(f) v = \int_G f(g) \pi(g)(v) d\mu(g).$$

Let us say here that $f \in L^1(G)$ is of trace class if for every unitary, absolutely irreducible representation $\pi$ of $G$, the operator $\pi(f)$ is of trace class. (side question: is there a name outside for this notion?).

Is it true that the space of trace class functions on $G$ is dense in $L^1(G)$? If not, is it true for a separable type I locally compact group ?

I know of certain larges classes of group for which it is true. Compact group and abelian locally compact groups are trivial examples, since all irreducible $\pi$'s are finite dimensional. Real and $p$-adic Lie group are other examples since smooth compact support functions are trace class (IIRC) with the usual meaning of smooth ($C^\infty$ in the real case, locally constant in the $p$-adic case), and are dense. Yet this is proved in a rather ad hoc way, for example in the real case (Duflo-Labesse) by showing that such a function $f$ is the convolution of two such functions $f_1$ and $f_2$, so that $\pi(f)=\pi(f_1)\pi(f_2)$ and $\pi(f_1), \pi(f_2)$ are Hilbert-Schmiddt. Also the adelic case (necessary for the global trace formula) follows from the real and $p$-adic case.

Yet I don't know if this is true in general, or at least for a large abstract class of locally compact group (defined with a property such as type I, not as a list of example such as "real Lie groups"), with if possible a uniform proof.

If the question is too difficult, or the answer is no, does that help if we ask the same question for functions that are reduced trace class, i.e. such that $\pi(f)$ is of trace class for all irreducible $\pi$'s in the reduced spectrum (i.e. in the support of the regular representation).

PS: Clearly, the condition of being a trace class does not change if we replace $L^1(G)$ by $C^\ast(G)$ with its natural norm. Hence one of the tag.

-
This is merely a comment for now as my brain is too flu-addled to check details properly, but I think that for separable Type I , the space of $f$ such that $\pi(f)$ is trace class for Plancherel-a.e. $\pi$ should be isomorphic, via the operator-valued Fourier transform, to the Fourier algebra of $G$. This is dense in $C_0(G)$ for the sup norm and hence dense in $L^1(G)$. – Yemon Choi Mar 11 '13 at 21:12
Also, IIRC, if one is looking for possible counterexample then one candidate might be the ax+b group (over the reals). The paper I have in mind is by Zep/Diep ams.org/mathscinet-getitem?mr=364539 although I think Gelfand+Neumark also did the relevant calculation. – Yemon Choi Mar 11 '13 at 21:18
Correction to my first comment: the space of such $f$ would contain the Fourier algebra but wouldn't in general be equal to it. However, for the OP's purposes, that still would imply density in $L^1(G)$. – Yemon Choi Mar 11 '13 at 21:19
Thanks a lot, Yemon. I need to read more about this notion of Fourier Algebra but that seems to be the right concept for answering my question. – Joël Mar 12 '13 at 13:19

The following is a suggestion how to prove a weak variant of the OP's original question affirmative.

Theorem: Let $G$ be a unimodular, seperable, type-I group. For every element $\phi$ in $C_c^\infty(G)$ (in the sense of Bruhat), the operator $\pi(\phi)$ is trace class for almost every unitary representation $\pi$ (with respect to the Plancherel measure).

Proof: The Plancherel theorem states for $\phi \in C_c^\infty(G)$, that $\pi( \phi \ast \phi^{\ast})$ is a trace class operator for a.e. $\pi$. The Dixmier-Malliavin theorem allows us to write general $f$ as a sum of convolution product $S \ast T^*$. A convolution product is the linear combination of four positive functions (polarisation identity), i.e.,

$$4 \cdot S \ast T^\ast = (S+T) \ast (S+T)^\ast - (S-T) \ast (S-T)^\ast$$ $$- i(S-iT) \ast (S-iT)^* + i(S+iT) \ast (S+iT)^*.$$ Applying that $\pi ( (S+T) \ast (S+T)^\ast ), \dots$ are trace class by Plancherel for a.e. $\pi$ yields the theorem. Q.e.d.

The following lemama should follow rather easily from the equivalent result of Lie groups:

Lemma: $C_c^\infty(G) \subset L^1(G)$ is a dense embedding.

Discussion of Dixmier-Malliavin: Every smooth, compactly supported functions is a finite sum of convolution products. This is due to Dixmier and Malliavin, originally proven for a Lie groups only.

Writing general $G$ as an projective limit of Lie groups, we obtain a notion of smooth, compactly supported functions on $G$. So the proof theorem Dixmier-Malliavin extends to locally compact groups. See also my Phd thesis http://webdoc.sub.gwdg.de/diss/2012/palm/palm.pdf, where I discuss this in chapter 5, in particular Theorem 5.2.1. Approximation via Lie groups is explained in chapter 4.

Discussion of the Plancherel theorem: A necessity of the Plancherel theorem (check e.g. http://www.encyclopediaofmath.org/index.php/Unitary_representation) is that the $tr\; \pi (f\ast f^*)$ is well-defined for every $f \in L^1(G) \cap L^2(G)$ and a.e. $\pi$ with respect to the Plancherel measure $\mu$. The Plancherel theorem states then for $G$ be a unimodular separable locally compact group of type I $$\int\limits_G |f(g)|^2 d g = \int\limits_{unitary \; dual} tr\; \pi( f \ast f^*) d \mu(\pi).$$ There exist extensions, which drop unimodular, type I or seperable as stated in the above link, but don't know how the extensions look like, so that's why I worked with rather special $G$ here.

You can furthermore put a (Fell-)topology on the unitary dual and so on, but I don't know wheter $\pi \mapsto \pi(\phi)$ will then be a continuous function!? Even a smooth structure via the approximation-by-lie-groups-business.

Using Dixmier-Malliavin plus the polarization identity you obtain for $f \in C_c^\infty(G)$ (Theorem 6.2.6, pg.83 in my thesis) the following variant of the Plancherel theorem $$f(1) = \int\limits_{unitary \; dual} tr\; \pi( f) d \mu(\pi).$$

For non-type-I groups, the Plancherel measure will not be unique (?), so I am sceptical about a direct generalization omitting that notion. I think that neither being unimodular nor seperable is a crucial hypothesis.

Discussion of reductive groups: For reductive groups over local fields, you can replace a.e. w.r.t. to Plancherel measure by every smooth, admissible representation, but I guess you are well aware of that. Some reps are not tempered and do not contribute to the Plancherel formula, so this far more. The notion of smooth in the sense I indicate above coincides with the usual one in the non-archimedean world/totally disconnected groups. Be careful, general Lie groups are not type I. For the adelic points of a reductive group over a global field, you need an argument along the lines of Is a reductive adelic group a Type I group?, but the situation is similar to that of reductive local groups.

-
Can you explicit a little bit? You say $G$ can be written as a projective limit of Lie groups ? what kind of group $G$ ? And why ? and where can I find your thesis ? – Joël Mar 11 '13 at 22:29
For the reference to Duflo-Labesse, you're right, I didn't remember it well and had no access to their paper. So they attribute the lemma to Dixmier-Malliavin, this I remember. And the result is "for any reductive real Lie group and any irreducible representation $\pi$, $\pi(f)$ is trace Class for $f$ smooth with compact support" ? – Joël Mar 11 '13 at 22:36
(2) Then what is connection with the question I ask? For $G$ a general locally compact group, is the space of compact support smooth function dense in L^1(G)? Fo such a function $f$ is it true to false that $\pi(f)$ is trace class for every unitary irreducible $\pi$ (it would be enough to say that $\pi(f)$ is Hilbert-Schmidt, by Dixmier-Malliavin, wouldn't it)? If the answers to this questions is yes then the answer to my question is yes in full generality, and I suppose you would have said it. So which one(s) have answer "no"? – Joël Mar 12 '13 at 13:00
(3) Then you suggest that I change my definition of trace class to... what exactly ? I presume you mean that I should say that $f$ is trace class if $\pi(f * f*)$ is trace class for Plancherel-almost all $\pi$. But I am not ready to do that. That changes completely my question. I agree to say that $f$ is of "almost reduced trace class" if $\pi(f)$ is trace class for all $\pi$ in the spectrum except for a set of measure 0 for the Plancherel measure. Since by definition, the support of the Plancherel measure is the reduced spectrum, "almost reduced trace class" implies "reduced trace class". – Joël Mar 12 '13 at 13:07
@Marc: I did not point out any such thing, since it did not seem particularly relevant to the original question. Please do not put words in my mouth – Yemon Choi Mar 12 '13 at 20:35