(Note : all the groups are locally compact topological groups)

I will prove the following theorem, and then prove that at least for Hilbert space this result is pretty much optimal:

**Theorem:** Let $G$ and $H$ be two postliminal groups, then every unitary irreducible representations of $G \times H$ over a Hilbert space is of the form $\rho \otimes \rho'$ where $\rho$ and $\rho'$ are unitary irreducible representations of $G$ and $H$ (over Hilbert spaces).

A post-liminal group (also called a type I group) is a group $G$ such that every for every unitary irreducible representation $\rho$ of $G$ on $H$, every compact operator on the Hilbert space $H$ can be approximated by linear combination of operator of the form $\rho(g)$. Equivalently, the group maximal $C^*$ algebra is type I/post liminal. Equivalently, all factor representation of the group generate a type I factor, etc... see Dixmier "C* algebra" for more information on post-liminal/type I $C*$ algebra and group...

All compact group are post-liminal, all commutative groups are post-liminal, all semi-simple lie group are post-liminal, nilpotent (lie ?) groups are post-liminal I, the class of postliminal group has some stability property (obvious exemple: a quotient of a post-liminal group is postliminal) etc...
But there exist an exemple (in dimension 5) of a solvable lie group which is not postliminal, free group are not postliminal, in fact all non-amenable group are not post-liminal etc...

The proof would go as follow:
Let $\rho$ be a irreducible unitary representation of $G \times H$ on a Hilbert space $T_0$. Let $A_G$ and $A_H$ the von Neuman algebra generated by $G$ and $H$ in $B(T_0)$.

$A_G$ and $A_H$ are factors: indeed if $u \in Z(A_G)$ then $u$ commutes to the action of $H$ because it is in $A_G$ and it commutes to the action of $G$ because it is in $A_H$ hence by Schur's lemma (which hold for unitary representation) it is a scalar.

Hence $A_G$ and $A_H$ are type I factor because $G$ and $H$ are type I algebras.

Then (this is the key step) using the decomposition theory of Dixmier's C* algebra chapter 5.4 one can decompose $T_0$ into $T_1 \otimes T_2$.

Indeed, $A_G$ is a type I factor acting on $T_0$ hence $T_0$ is isomorphic to $T_1 \otimes T_2$ with $A_G = B(T_1)$ and as the action of $A_H$ is in the commutant of $A_G$ it is in $B(T_2)$. One then easily see that $T_1$ and $T_2$ are irreducible representation of $G$ and $H$:

if $u$ is an endomorphism of $T_1$ commuting to the action of $G$ then it clearly also (when seen as acting on $T_0$) it clealry commutes to the action of $G \times H$ and hence is a scalar.

This concludes the proof of the theorem.

In fact one can state the following:

**Theorem:** let $G$ and $H$ be two groups and $V$ be a unitary irreducible representation of $G \times H$ then $V$ is of the form $\rho \otimes \rho'$ if and only if the actions of $G \times \{1\}$ and $\{1 \} \times H$ on $V$ (respectively) generates type I von Neumann algebras.

Indeed one direction is exactly the previous proof, and conversely if the representation can be decomposed, then (using the notation of the previous proof) $A_G$ is isomorphic to $B(T_1)$ and hence is a type I factor.

**Corollary:**
Let $G$ be a group, then the following conditions are equivalent:

proof:
If $G$ is post-liminal then this is just the first theorem.
Conversely, assume $G$ is not post liminal, then there exists a Hilbert space $H$ with a unitary representation of $G$ that generate a type II or type III *factor* $A_G$.

Now considering the $l^2$ representation of $A_G$ one obtains a representation $H'$ of $A_G$ such that $End_{A_G}(H) = (A_G)^{op}$. (this refers to Tomita-Takesaki theory and has nothing to do with the regular representation of $G$)

One can make $G \times G$ acts on $H'$: the first component acts though $G \rightarrow A_G$ and the second component to $G \rightarrow A_G^{op}$ (sending $g$ to $g^{-1}$).

$H'$ is irreducible: indeed an endomorphism of $H'$ commuting with the action og $G \times G$ commute with the action of $A_G$, and of $(A_G)'$ (the commutant of $A_G$) hence by the double commutant theorem it belong to the center of $A_G$ which is reduce to the scalar as $A_G$ has been assume to be a factor.

By the second theorem $H'$ is a rep of $G \times G$ that cannot be decomposed: indeed the algebra generated by $G$ is $A_G$ and is of type II or type III by assumption.