Unitary dual of the motion group $M(n)$, for $n> 2$

Question

The motion group of $\mathbb R^2$, noted by $G=M(2)$ is the semi-direct product of $\mathbb R^2$ with the special orthogonal group $K = SO(2)$. A well known fact is that the unitary dual $\hat{G}$, of $G$ is precisely the collection [see, Sugiura, M.: Unitary Representations and Harmonic Analysis. Kodansha Scientific books, Tokyo (1975), p: 165]: $$\{\pi_\lambda,\, \lambda\in>0 \}\cup\{\chi_n: \, n\in \mathbb Z \},$$ where $[\pi_{\lambda}(z,e^{i\alpha})u](s) = e^{i\lambda \left<z,e^{is}\right>} u(s-\alpha)$ and where $\chi_n(z,e^{i\alpha})=e^{in\alpha}$.

Now, the infinite dimensional, unitary irreducible representations of the motion group $M(n)$ of $\mathbb R^n$ is given by [see, N. I. Vilenkin, (1978). Special functions and the theory of group representations (Vol. 22). American Mathematical Soc]:

For each $\lambda>0$, a unitary representation $\pi_{\lambda}$ of $M(n)$ realized on $L^2(S^{n-1})$ is \begin{align*} (*) \quad [\pi_{\lambda}(a,k)u](\xi) = e^{i\lambda \left<x,\xi\right>}\, u(k^{-1}\xi), \end{align*} for $(x,k)\in M(n)=\mathbb R^n \rtimes SO(n)$ and $u \in L^2(S^{n-1})$.

Apart from these family, we have another family $\{\chi_m, m \in ? \}$, of one dimensional unitary representations of $M(n)$, which ? Is what it is $\chi_n(x,k)=e^{i m.k}, m=(m_1, \dots, m_n) \in \mathbb Z^{n} $ with $m.k= m_1 k_1 + \dots + m_n k_n$. Or what ?

Thank you in advance

Answer 1

This is done by Mackey theory and discussed in many places, e.g. see Lipsman (1974, page 72) for an explicit list. In short, there are two series:

1. Your sought $\chi$’s: all (finite dimensional) irreps of $\mathrm{SO}(n)=M(n)/\mathbf R^n$ lifted from that quotient;

2. Representations induced from irreps $e^{i\langle\lambda,\cdot\rangle}\otimes\sigma$ of the stabilizer $\mathbf R^n\rtimes\mathrm{SO}(n-1)$ of a pole in the action of $M(n)$ on $S^{n-1}$ (via $\mathrm{SO}(n)$ again). For nontrivial $\sigma$ these act not in $L^2(S^{n-1})$ but in $L^2$ sections of (associated) vector bundles on $S^{n-1}$.

Best understood, in my opinion, from the point of view of Kirillov’s orbit method: (1) are attached to the compact coadjoint orbits of $M(n)$, which are themselves pulled back from the quotient $\mathrm{SO}(n)$; (2) to the noncompact orbits, which are themselves “symplectically induced” from coadjoint orbits of the stabilizer.