I understand it for finite groups now, and I suspect that it's similar for semisimple Lie groups.

These statements are equivalent, and true for finite groups:

* The adjunction $\operatorname{Res} \dashv \operatorname{CoInd}$ is monadic (for compact $G/H$), its monad is $\mathrm{L}^2(G/H) \otimes -$.
* $H$-representations are $L^2(G/H)$-representations internal to the category of $G$-representations.
* $H$-representations are $G$-equivariant maps $\mathrm{L}^2(G/H) \otimes \rho \to \rho$, where $\rho$ is a $G$-representation, satisfying certain associativity and unit laws.