Unitary dual of the motion group $M(n)$, for $n> 2$

The motion group of $\mathbb R^2$, noted by $G=M(2)$ is the semi-direct product of $\mathbb R^2$ with the special orthogonal group $K = SO(2)$. A well known fact is that the unitary dual $\hat{G}$, of $G$ is precisely the collection [see, Sugiura, M.: Unitary Representations and Harmonic Analysis. Kodansha Scientific books, Tokyo (1975), p: 165]: $$\{\pi_\lambda,\, \lambda\in>0 \}\cup\{\chi_n: \, n\in \mathbb Z \},$$ where $[\pi_{\lambda}(z,e^{i\alpha})u](s) = e^{i\lambda \left<z,e^{is}\right>} u(s-\alpha)$ and where $\chi_n(z,e^{i\alpha})=e^{in\alpha}$.

Now, the infinite dimensional, unitary irreducible representations of the motion group $M(n)$ of $\mathbb R^n$ is given by [see, N. I. Vilenkin, (1978). Special functions and the theory of group representations (Vol. 22). American Mathematical Soc]:

For each $\lambda>0$, a unitary representation $\pi_{\lambda}$ of $M(n)$ realized on $L^2(S^{n-1})$ is \begin{align*} (*) \quad [\pi_{\lambda}(a,k)u](\xi) = e^{i\lambda \left<x,\xi\right>}\, u(k^{-1}\xi), \end{align*} for $(x,k)\in M(n)=\mathbb R^n \rtimes SO(n)$ and $u \in L^2(S^{n-1})$.

Apart from these family, we have another family $\{\chi_m, m \in ? \}$, of one dimensional unitary representations of $M(n)$, which ? Is what it is $\chi_n(x,k)=e^{i m.k}, m=(m_1, \dots, m_n) \in \mathbb Z^{n}$ with $m.k= m_1 k_1 + \dots + m_n k_n$. Or what ?

This is done by Mackey theory and discussed in many places, e.g. see Lipsman (1974, page 72) for an explicit list. In short, there are two series:

1. Your sought $\chi$’s: all (finite dimensional) irreps of $\mathrm{SO}(n)=M(n)/\mathbf R^n$ lifted from that quotient;

2. Representations induced from irreps $e^{i\langle\lambda,\cdot\rangle}\otimes\sigma$ of the stabilizer $\mathbf R^n\rtimes\mathrm{SO}(n-1)$ of a pole in the action of $M(n)$ on $S^{n-1}$ (via $\mathrm{SO}(n)$ again). For nontrivial $\sigma$ these act not in $L^2(S^{n-1})$ but in $L^2$ sections of (associated) vector bundles on $S^{n-1}$.

Best understood, in my opinion, from the point of view of Kirillov’s orbit method: (1) are attached to the compact coadjoint orbits of $M(n)$, which are themselves pulled back from the quotient $\mathrm{SO}(n)$; (2) to the noncompact orbits, which are themselves “symplectically induced” from coadjoint orbits of the stabilizer.

• @ Francois Ziegler, I would like to know an explicit expression of the unitary dual of $M(n)$ as it is given for the case $M(2)$. Jan 7 '18 at 19:01
• I added details; Lipsman’s list is “explicit” to the extent that $\mathrm{SO}(n-1)\hat{\ }$ is. Jan 7 '18 at 19:41
• Dear Francois Ziegler, I would like to realize it on $L^2(S^{n-1})$. Then, according to N. I. Vilenkin $\pi_\lambda$ is given by $(*)$ above and I'm looking for the expression of $\chi$ (always on $L^2(S^{n-1})$) ? Jan 7 '18 at 21:57
• @Z.Alfata Then you aren’t talking about the whole unitary dual. IIRC, Vilenkin limits himself to irreps with a $K$-fixed vector ($K=\mathrm{SO}(n)$); those are called various names and, correct me if I’m wrong, should here be: in (1) just the trivial representation; in (2) those with trivial $\sigma$. The latter act in $\smash{L^2(S^{n-1})}$ as you desire, with $K$-fixed vector the constant 1. Jan 7 '18 at 23:22
• @ Francois Ziegler, in Bhowmik,Sen - JFAA 2017, p: 7-8 "the finite dimensional unitary representations of $SO(n)$ also yield finite dimensional unitary representations of $M(n)$'. Then, the expression the finite dimensional unitary representations of $SO(n)$ is a character or what ? Jan 24 '18 at 12:33