Generalization of a theorem of Burnside to non-compact groups The following two theorems are often attributed to Burnside:
Theorem Let $G$ and $H$ be compact groups. Then the irreducible representations of $G\times H$ are precisely the representations $\pi\otimes \sigma$ where $\pi$ and $\sigma$ are irreducible representations of $G$ and $H$, respectively.
Corollary Let $(\pi, V)$ be an irreducible representation of the compact group $G$. Then every linear transformation of $V$ is of the form


*

*$\sum_i a_i\pi(g_i)$ (finite sum) for some $a_i\in \mathbb{C}$ and $g_i\in G$. 

*$\pi(f)$ for some continuous function $f$.


The Theorem is a fairly simple consequence of orthogonality relation for characters and the Corollary is an application of the Theorem to $\pi\otimes \pi^*$.
My questions are: 
(1) how does this beautiful result of Burnside generalize to non-compact (Lie) groups? (I can imagine that both results fail to hold although I haven't cooked up any counterexamples yet.)
(2) does the Corollary hold when $V$ is finite dimensional?
 A: (Note : all the groups are locally compact topological groups)
I will prove the following theorem, and then prove that at least for Hilbert space this result is pretty much optimal: 
Theorem: Let $G$ and $H$ be two postliminal groups, then every unitary irreducible representations of $G \times H$ over a Hilbert space is of the form $\rho \otimes \rho'$ where $\rho$ and $\rho'$ are unitary irreducible representations of $G$ and $H$ (over Hilbert spaces).
A post-liminal group (also called a type I group) is a group $G$ such that every for every unitary irreducible representation $\rho$ of $G$ on $H$, every compact operator on the Hilbert space $H$ can be approximated by linear combination of operator of the form $\rho(g)$. Equivalently, the group maximal $C^*$ algebra is type I/post liminal. Equivalently, all factor representation of the group generate a type I factor, etc... see Dixmier "C* algebra" for more information on post-liminal/type I $C*$ algebra and group...
All compact group are post-liminal, all commutative groups are post-liminal, all semi-simple lie group are post-liminal, nilpotent (lie ?) groups are post-liminal I, the class of postliminal group has some stability property (obvious exemple: a quotient of a post-liminal group is postliminal) etc...
But there exist an exemple (in dimension 5) of a solvable lie group which is not postliminal, free group are not postliminal, in fact all non-amenable group are not post-liminal etc...
The proof would go as follow:
Let $\rho$ be a irreducible unitary representation of $G \times H$ on a Hilbert space $T_0$. Let $A_G$ and $A_H$ the von Neuman algebra generated by $G$ and $H$ in $B(T_0)$.
$A_G$ and $A_H$ are factors: indeed if $u \in Z(A_G)$ then $u$ commutes to the action of $H$ because it is in $A_G$ and it commutes to the action of $G$ because it is in $A_H$ hence by Schur's lemma (which hold for unitary representation) it is a scalar.
Hence $A_G$ and $A_H$ are type I factor because $G$ and $H$ are type I algebras.
Then (this is the key step) using the decomposition theory of Dixmier's C* algebra chapter 5.4 one can decompose $T_0$ into $T_1 \otimes T_2$.
Indeed, $A_G$ is a type I factor acting on $T_0$ hence $T_0$ is isomorphic to $T_1 \otimes T_2$ with $A_G = B(T_1)$ and as the action of $A_H$ is in the commutant of $A_G$ it is in $B(T_2)$. One then easily see that $T_1$ and $T_2$ are irreducible representation of $G$ and $H$:
if $u$ is an endomorphism of $T_1$ commuting to the action of $G$ then it clearly also (when seen as acting on $T_0$) it clealry commutes to the action of $G \times H$ and hence is a scalar.
This concludes the proof of the theorem.
In fact one can state the following:
Theorem: let $G$ and $H$ be two groups and $V$ be a unitary irreducible representation of $G \times H$ then $V$ is of the form $\rho \otimes \rho'$ if and only if the actions of $G \times \{1\}$ and $\{1 \} \times H$ on $V$ (respectively) generates type I von Neumann algebras.
Indeed one direction is exactly the previous proof, and conversely if the representation can be decomposed, then (using the notation of the previous proof) $A_G$ is isomorphic to $B(T_1)$ and hence is a type I factor.
Corollary:
Let $G$ be a group, then the following conditions are equivalent:


*

*$G$ is postliminal.

*Every (unitary) irreducible representation of $G \times G$ decompose as a tensor product of two irreducible representations of $G$.
proof:
If $G$ is post-liminal then this is just the first theorem.
Conversely, assume $G$ is not post liminal, then there exists a Hilbert space $H$ with a unitary representation of $G$ that generate a type II or type III factor $A_G$.
Now considering the $l^2$ representation of $A_G$ one obtains a representation $H'$ of $A_G$ such that $End_{A_G}(H) = (A_G)^{op}$. (this refers to Tomita-Takesaki theory and has nothing to do with the regular representation of $G$)
One can make $G \times G$ acts on $H'$: the first component acts though $G \rightarrow A_G$ and the second component to $G \rightarrow A_G^{op}$ (sending $g$ to $g^{-1}$).
$H'$ is irreducible: indeed an endomorphism of $H'$ commuting with the action og $G \times G$ commute with the action of $A_G$, and of $(A_G)'$ (the commutant of $A_G$) hence by the double commutant theorem it belong to the center of $A_G$ which is reduce to the scalar as $A_G$ has been assume to be a factor.
By the second theorem $H'$ is a rep of $G \times G$ that cannot be decomposed: indeed the algebra generated by $G$ is $A_G$ and is of type II or type III by assumption.
A: I'm going to leave two answers to this question. This one will point out that, in the finite dimensional case, everything is fine. The other will point out that, in the Hilbert space case, there seem to be relationships to the invariant subspace problem.
I'll write out the proofs of these results for finite dimensional vector spaces in enough detail to show that we never use compactness of $G$. 
Conventions:  I abbreviate "representation" to "rep" and "irreducible representation" to "irrep". The zero representation is not irreducible.
Let $W$ be a finite dimensional irrep of $G \times H$ over $\mathbb{C}$. For any irrep $V_i$ of $H$, we have a natural map 
$$\mathrm{Hom}_H(V, W) \otimes V \to W. \quad (\ast)$$
The image of $(\ast)$ a $G \times H$ subrep, so it is either $W$ or $(0)$. Moreover, any $H$-rep contains a sub-irrep (first use of finite dimensionality), so there is some $V$ for which the image is $W$.
We first claim that, for this $V$, we have $W \cong V^{\oplus r}$ as $H$-reps for some $r$. Proof: $(\ast)$ gives us a surjection $V^{\oplus s} \to W$ for some $s$ (second use of finite dimensionality, but could be replaced by the axiom of choice and letting $s$ be an ordinal). We write the left hand side as $\phi: V_1 \oplus V_2 \oplus \cdots \oplus V_s \to W$. For each $i$, the map $V_i \to W/\phi(V_1 \oplus \cdots \oplus V_{i-1})$ is either $0$ or injective (since $V_i$ is irreducible). Let $I$ be the set of $i$ for which this map is injective. Then $\phi: \bigoplus_{i \in I} V_i \to W$ is an isomorphism, so $W \cong V^{\oplus \#(I)}$. (This argument is basically putting matrices into row echelon form, written very abstractly.) This proves the claim.
Now, by Schur's lemma (third use of finite dimensionality, and the one which strikes me as hardest to avoid), $\mathrm{Hom}_H(V, V) = \mathbb{C}$. So $\dim \mathrm{Hom}_H(V, W) = r$ for the same integer $r$ as above. We deduce that $(\ast)$ is an isomorphism. This proves your Theorem.
Corollary 1 follows as before.
To talk about corollary 2, we need a topology on $G \times H$ (so we can talk about continuity) and a measure (so we can talk about integration and define $\pi(f) = \int_G f(g) \pi(g)$). I don't want to try to get the details right, but I'll point you to the modern formulation of Burnside's theorem: If $V$ is a finite dimensional $\mathbb{C}$ vector space, and $A \subseteq \mathrm{End}(V)$ is an algebra such that $V$ is an irreducible $A$-module, then $A = \mathrm{End}(V)$. Apply this to the algebra of all matrices of the form $\int_G f(g) \pi(g)$, where $f$ is any compactly supported function $G \to \mathbb{C}$. (This is an algebra because of convolution.) With any reasonable topology, you should be able to make bump functions and show that, if $V$ is irreducible as a $G$-rep, then it is irreducible as an $A$-module.
A: This is one of two answers to this question. The other one addresses the finite dimensional case. This one points out that the Hilbert space case seems to involve the invariant subspace problem. Disclaimer: I am really bad at functional analysis.
Suppose that $W$ is a Hilbert space of dimension $>1$ and $T: W \to W$ is an invertible operator with no invariant subspace.  (If the ISP has counter-examples, it has invertible counter-examples; see the comment of Simon Henry below.) 
Let $\mathbb{Z} \times \mathbb{Z}$ act on $W$ by $(i,j) : w \to T^{i+j}(w)$. Then $W$ is an irreducible $\mathbb{Z} \times \mathbb{Z}$ rep, but it is not of the form $U \otimes V$ for $\mathbb{Z}$-reps $U$ and $V$. Proof: Suppose it were. Choose a nonzero vector $u \in U$. Then $u \otimes V$ would be an $H$-subrep, and hence an invariant subspace. So $u \otimes V = W$ and $\dim U=1$. Similarly, $\dim V=1$. But then $\dim W = 1$, a contradiction.
Of course, this means that the statement should be false for more general things like Banach spaces, where the invariant subspace problem fails.
