Dimension of intertwiner space: finite-dimensional representations of compact groups

Question

Let $G$ be a compact group with finite-dimensional, real representations $\phi$ and $\psi$ on $V$ and $W$ respectively. (e.g. $V = \mathbb{R}^m$, $W = \mathbb{R}^n$.) Is it true that, as is the case for finite groups, the dimension of the intertwiner space of the two representations is equal to the inner product of the characters? That is, do we still have

$$ \dim \text{Hom}_G(V, W) = \int_G \text{d}\lambda(g) \text{Tr}(\phi(g))\text{Tr}(\psi(g)) $$ where $\lambda$ is the Haar measure on $G$?

As an aside, do the proposed representations always exists? Do they exist if I insist they are unitary/orthogonal?

Answer 1

Yes, this formula holds, and such representations exist.

If $\mathbb K$ is either $\mathbb R$ or $\mathbb C$ and $G$ is a compact topological group with $\mathbb K$-representations $V$ and $W$, then $Hom_{\mathbb K}(V,W)$ is another representation of $G$ satisfying $$ Hom_G(V,W) = Hom_{\mathbb K}(V,W)^G.$$ For any representation $U$ of $G$, we have the formula $$ \dim U^G = \int_G tr(g|_U) d\lambda(g).$$ This follows from considering the integral operator $$ e_U = \int_G g|_U d\lambda(g),$$ which is an idempotent in $End(U)$ projecting onto the invariants $U^G$. Hence $tr(e_U) = \dim U^G$.

If $V$ and $W$ have characters $\phi$ and $\psi$, then the trace of $g \in G$ on $Hom(V,W)$ is $\phi(g^{-1})\psi(g)$. For compact groups, the representation is conjugate to a unitary representation, and hence $\phi(g^{-1}) = \overline{\phi(g)}$. Assuming your representation is real, this yields your formula.

Finite-dimensional unitary representations over $\mathbb C$ exist by the Peter-Weyl theorem. Restricting scalars to $\mathbb R$ gives real finite-dimensional orthogonal representations.