The conclusion of the part of the Peter-Weyl theorem stating that unitary representations of the group split as an orthogonal direct sum of finite dimensional ones is false if the group is not compact. There are even non-compact groups which simply do not admit any unitary finite dimensional representations, like the metaplectic group $Mp_{2p}$, which is the double cover of the symplectic group $\mathrm{Sp}_{2p}$$\mathrm{SL}(2,\mathbb R)$, which has, on the other hand, irreducible infinite dimensional representations.