I would think that this result would go all the way back to Frobenius. Anyway, the proof seems easy enough:

Let $V$ be the trivial $\Gamma$-module. We want to show that $\dim H_\Gamma=\dim\mathrm{Hom}_{\Gamma}(V,H)\leq 1$. To this end, note that by Frobenius reciprocity,
$$
\mathrm{Hom}_{\Gamma}(V,H)\cong\mathrm{Hom}_{G\rtimes\Gamma}(\mathrm{Ind}_{\Gamma}^{G\rtimes\Gamma}V,H)
$$
so it suffice to show that the right-hand side is at most 1-dimensional. As $H$ is irreducible, this amounts to showing that every irreducible summand of $\widehat{V}:=\mathrm{Ind}_{\Gamma}^{G\rtimes\Gamma}V$ appears with multiplicity 1 (by Schur's lemma). 

Well, this is easy since $\widehat{V}\cong\mathbb{C}G$ as a $G$-module and $\mathbb{C}G$ decomposes as a direct sum of $|G|$ non-isomorphic $G$-modules. On the other hand, if $U\oplus U\leq\widehat{V}$ for some irreducible $G\rtimes\Gamma$-module $U$, then decomposing each $U$ as a $G$-module shows that there are multiple copies of the same irreducible $G$-module in $\widehat{V}$ upon restriction to $G$. This is a contradiction.

Hence, $\dim H_\Gamma\leq 1$.