A morphism intertwining two induced representations

TL;DR:
Given representations $$D,\Lambda$$ of subgroups $$K,Q$$ of a Lie group $$G$$, is it true that every intertwining operator $$T$$ between the resulting induced representations of $$G$$ can be written $$(T\varphi)(g) = \int_G t(g^{-1} g')\varphi(g')\,dg' \tag1$$ for some function $$t$$ on $$G$$?

LONG VERSION:

I placed this question at the Mathematics StackExchange. Although I presumed that the question is simple if not trivial, no one was able to help me there. So I am now elevating it to the MathOverflow.

Consider the space $$\,{\cal{L}}^G\,$$ of all continuous functions $$\,G\longrightarrow{\cal{L}}\,$$ mapping a Lie group $$\,G\,$$ into a vector space $$\,{\cal{L}}\,$$.

Assume that $$\,G\,$$ has two proper subgroups: $$K\,,~Q~<~G~~,$$ whose representations, $$\,D(K)\,$$ and $$\,\Lambda(Q)\,$$, are acting in $$\,{\cal{L}}^G\,$$.

Consider two subspaces of $$\,{\cal{L}}^G\,$$. One subspace, $${\mbox{Map}}_K(G,\,{\cal{L}})\,=\,\left\{\,\varphi\,\right\}~~,$$ comprises the vector functions $$\,\varphi\,$$ obeying the equivariance condition $$\varphi(g\, k)~=~D^{-1}(k)~\varphi(g)~,~~~k\,\in\, K~~.$$
In this subspace, $$\,D(K)\,$$ is induced to a representation of $$\,G\,$$, denoted by $$U^{(D)}\,\equiv\,D(K)\,\uparrow\, G$$ and implemented with $$U^{(D)}_g\,\varphi(g^{\,\prime})~=~\varphi(g^{-1}\, g^{\,\prime})~~.$$

Another subspace, $${\mbox{Map}}_Q(G,\,{\cal{L}})\,=\,\left\{\,\psi\,\right\}$$ will comprise the functions $$\,\psi\,$$ satisfying $$\psi(g\, q)~=~\Lambda^{-1}(q)~\psi(g)~,~~~q\,\in\, Q~~.$$ In this subspace, $$\,\Lambda(Q)\,$$ is induced to a representation of $$\,G\,$$, denoted by $$U^{(\Lambda)}\,\equiv \,\Lambda(Q)\,\uparrow\, G$$ and implemented with $$U^{(\Lambda)}_g\,\psi(g^{\,\prime})~=~\psi(g^{- 1}\, g^{\,\prime})~~.$$

While both $$\,U^{(D)}\,$$ and $$\,U^{(\Lambda)}\,$$ are realised via left translations, they are different representations, as they are acting in subspaces defined by different subsidiary conditions.

For convenience, we summarise this in the table: $$\varphi(g\, k)=D^{-1}(k)\,\varphi(g)\,,~k\in K \quad \quad \psi(g\, q)=\Lambda^{-1}(q)\,\psi(g)\,,~q\in Q$$ $$U^{(D)}\,\equiv\,D(K)\,\uparrow\, G \qquad \qquad \qquad U^{(\Lambda)}\,\equiv \,\Lambda(Q)\,\uparrow\, G$$ $$U^{(D)}_g\,\varphi(g^{\,\prime})~=~\varphi(g^{-1}\, g^{\,\prime}) \quad \quad \qquad U^{(\Lambda)}_g\,\psi(g^{\,\prime})~=~\psi(g^{- 1}\, g^{\,\prime})~$$

Our goal is to describe the space $$\,\left[\, D(K)\,\uparrow\, G\,,~\Lambda(Q)\,\uparrow\, G \,\right]\,$$ of the morphisms $$\,\psi\,=\,\hat{T}\,\varphi\,$$.

QUESTION:

How to prove that the most general form of a morphism is $$\psi(g)~=~(\hat{T}\,\phi)(g)~=~\int_G t(g^{-1}\, g^{\,\prime})\,\varphi(g^{\,\prime})\,dg^{\,\prime}~~,\qquad\qquad\qquad(1)$$ where $$\,dg\,$$ is an invariant measure on $$\,G\,$$.

PS.

As an aside, I would mention that for the equivariance conditions to be satisfied the kernel must obey one more condition: $$t(qgk) = \Lambda(q) t(g) D(k)~~,~~~q\in Q\,,~~g\in G\,,~~k\in K~~.$$ This, however, is the next theorem; and I don't want to go there until the basic property (1) is proven.

• I added your requested tags, and also a TL;DR as I think the question could benefit from being much shorter. – Francois Ziegler Nov 3 '19 at 16:29
• @FrancoisZiegler Thanks again! Would you recommend me to remove the redundant proof attempt? – Michael_1812 Nov 3 '19 at 18:14

(1) is certainly false in general, but some things like it hold at least formally. See e.g. “Motivation” in Knapp (1986, §VII.3), which will send you to precise statements like Mackey’s Intertwining Number Theorem for representations induced from open subgroups (1951, Theorems 3 and 3’) (also found in Curtis-Reiner (1962, p. 327) for finite $$G$$), or Bruhat’s version for Lie groups, using distribution kernels (1956, pp. 160, 167, 171).

Edit: To answer your added comments, consider the case: $$G$$ is discrete, $$D$$ and $$\Lambda$$ are characters (1‑dimensional representations). Then the module $$\mathrm{ind}_K^GD$$ (where $$G$$ acts by $$(g\varphi)(g')=\varphi(g^{-1}g')$$) has the easily checked “reproducing kernel” property that its members satisfy $$\varphi(g)=(gD^\bullet,\varphi) \tag i$$ where $$(\cdot,\cdot)=$$ inner product1 and $$D^\bullet\in\mathrm{ind}_K^GD$$ is the function (cyclic vector) equal to $$\overline D$$ in $$K$$ and zero outside. Likewise for $$\mathrm{ind}_Q^G\Lambda$$, mutatis mutandis. So if $$T\in\text{Hom}_G(\mathrm{ind}_K^GD,\mathrm{ind}_Q^G\Lambda)$$ and we let $$t:=T^*\Lambda^\bullet$$ (star $$=$$ adjoint), we get (except for a complex conjugate in the notation) your desired result (1): $$(T\varphi)(g) =(g\Lambda^\bullet,T\varphi) %=(gT^*\Lambda^\bullet,\varphi) =(gt,\varphi) =\sum_{g'K\in G/K}\overline t(g^{-1}g')\varphi(g'). \tag{ii}$$ Next one observes that (ii) applied to $$\varphi=D^\bullet$$ gives $$TD^\bullet=t^\vee$$ where $$f^\vee(g):=\overline f(g^{-1})$$. So $$t$$ is in $$(\mathrm{ind}_K^GD)\cap(\mathrm{ind}_Q^G\Lambda)^\vee$$, hence it satisfies the relation (complex conjugate of yours) $$t(q^{-1}gk) = D(k^{-1})t(g)\Lambda(q). \tag{iii}$$ Such a function is determined by one value per double coset $$QgK$$. Moreover, as one sees by putting $$q=gkg^{-1}$$ in (iii), this value must vanish unless $$k\mapsto\Lambda(gkg^{-1}) \style{font-family:sans-serif}{\text{ coincides with }} D \style{font-family:sans-serif}{\text{ on }} K\cap g^{-1}Qg. \tag{iv}$$ (This is the “intertwining two representations defined on a subgroup” you ask about.) Summing up, we get that $$\dim(\text{Hom}_G(\mathrm{ind}_K^GD,\mathrm{ind}_Q^G\Lambda))\leqslant \sharp\{$$double cosets $$QgK\in Q\backslash G/K$$ satisfying (iv)$$\}$$, which is the intertwining number theorem. It goes back to Shoda (1933, p. 251); adapting it to non-discrete groups (with sums replaced by integrals, kernels by distributions, etc.) raises hard analytic problems famously solved in some cases by Knapp, Kunze, Stein, etc.

1. $$(\varphi_1,\varphi_2):=\sum_{gK\in G/K}\overline{\varphi_1(g)}\varphi_2(g)$$, where bar $$=$$ complex conjugate and there is one term per coset (within which choice of $$g$$ doesn’t matter, thanks to the equivariance condition).

• Dear Francois, I am really grateful to you for this Edit. Will read and digest this. Meanwhile, I also have found this article by James Arthur from Toronto, where he warns the reader that the integral for the intertwiner, generally, does not converge: math.toronto.edu/arthur/pdf/Induced_representations.pdf – Michael_1812 Nov 3 '19 at 13:40
• Yes, or see also the discussion in Warner (1972, §5.0, especially p. 364, top): the problem with (e.g.) spaces of continuous functions is that they are not “nuclear”, so operators between them won’t all have your “kernel” form $\smash{\varphi\mapsto\int K(\cdot,g')\varphi(g')\,dg'}$; hence one works with spaces of “$C^\infty$ vectors” and then distribution kernels. – Francois Ziegler Nov 4 '19 at 15:37
• Dear Francois, thank you for this reference also. I am now looking in Warner's book. Given my lack of a bona fide mathematical education, I have to ask you a silly question. Is my understanding correct that $C_c(G)$ is the space of all continuous functions $f(g)$? Then what is $C_c^{\infty}(G)$? – Michael_1812 Nov 4 '19 at 18:50
• Also, could you please help me in understanding the notation in your main answer? I see that $D$ is a 1-dim representation of $K<G$. What is $\bar{D}$? Now, I see that $D^{\bf{\LARGE{\centerdot}}}$ is a function which is nonzero only in $K$. Then, if $\,gD^{\bf{\LARGE{\centerdot}}}\,$ is the function $\,F(g') = D^{\bf{\LARGE{\centerdot}}}(g^{-1}g')\,$, why does its dot product with $\varphi$ render us exactly $\varphi(g)$? – Michael_1812 Nov 4 '19 at 19:07
• $C_c$ and $C_c^\infty$ $=$ continuous (resp. smooth) functions with compact support; $gD^\bullet$ is zero outside coset $gK$, so $$(gD^\bullet,\varphi)=\sum_{g'K\in G/K}\overline{(gD^\bullet)(g')}\varphi(g')=\overline{(gD^\bullet)(g)}\varphi(g)=\varphi(g).$$ – Francois Ziegler Nov 4 '19 at 19:47

I have now found a remarkably simple proof of the fact that the variables $$\,g\,$$ and $$\,g^{\,\prime}\,$$ enter $$\,t(g\,,\, g^{\,\prime})\,$$ through convolution: $$t(g\,,\, g^{\,\prime})\;=\;t(g^{-1}\, g^{\,\prime})~~.\qquad\qquad\qquad (*)$$ To prove this, we should use the definition of an intertwiner, $$U_{g_1}^{(A)}\,T\,\varphi~=~T\,U^{(D)}_{g_2}\,\varphi~~.\qquad\qquad\qquad (**)$$ Its left-hand side can be written down as $$(\, U_{g_1}^{(A)}\, T\,\varphi)\,(g_2)\,=\,(T\,\varphi)\,(g_1^{-1}g_2) \,=\,\int t(g_1^{-1}g_2\,,\,g^{\,\prime})\,\varphi(g^{\,\prime})\,dg^{\,\prime}\,~.$$ Its right-hand side can be processed as $$(T\,U_{g_1}^{(D)}\,\varphi)\,(g_2)$$ $$=\,\int t(g_2\,,\,g^{\,\prime\prime})\,(U_{g_1}^{(D)}\varphi)\,(g^{\,\prime\prime})\,dg^{\,\prime\prime} \,=\,\int t(g_2\,,\,g^{\,\prime\prime})\,\varphi(g^{-1}\, g^{\,\prime\prime})\,dg^{\,\prime\prime}$$ $$=\int t(g_2,\,g_1(g_1^{-1}\, g^{\prime\prime}))\,\varphi(g^{-1}g^{\prime\prime})\,dg^{\prime\prime} =\int t(g_2,\,g_1 g^{\prime})\,\varphi(g^{\prime})\, dg^{\prime}~~,$$ where we defined $$\,g^{\,\prime}=\, g_1^{-1}\, g^{\,\prime\prime}\,$$ and assumed the measure left-invariant, $$\,dg^{\,\prime}=\, d(g_1^{-1} g^{\,\prime\prime}) \,=\, dg^{\,\prime\prime}\,$$.

If we plug the right-hand sides of the former and latter formulae in equation (**), we shall arrive at $$t(g_1^{-1}g_2\,,\,g^{\,\prime})~=~t(g_2\,,\,g_1 g^{\,\prime})~~.$$ If we now set $$\,g_2=1\,$$ and $$\,g=g_1^{-1}\,$$, the above formula will shape into $$t(g\,,\,g^{\,\prime})~=~t(1\,,\,g^{-1} g^{\,\prime})~~.$$ We now can rename $$\,t(1\,,\,g^{-1} g^{\,\prime})\,$$ as $$\,t(g^{-1} g^{\,\prime})\,$$ and regard (*) proven: $$\,t(g\,,\, g^{\,\prime})\,=\, t(g^{-1}\, g^{\,\prime})\,$$. Accordingly, $$\psi(g)\,=\,(\hat{T}\,\phi)(g)\,=\,\int t(g^{-1}\,g^{\,\prime})\,\varphi(g^{\,\prime})\,dg^{\,\prime}\;\;.$$ My understanding is that this integral may, in principle, diverge even when the representations are defined on nuclear functions. (Please correct me if I am wrong.) In such situations, one has to establish analytic continuation, i.e. to define this integral in the complex domain, where is does converge to an analytic function.