This is not an answer, but it's a bit too long for a comment.
If I'm not mistaken, any group $G$ which embeds densely into a compact Hausdorff group $H$ has the property that it is finite if and only if it has finitely many irreducible representations. This includes the residually finite groups, which embed densely into their profinite completions. To see this, use the fact that by Peter-Weyl $L^2(H)$ decomposes as a direct sum $\bigoplus_i n_i V_i$ where $V_i$ are the finite-dimensional unitary irreducible representations of $H$ and $n_i = \dim V_i$. It follows that there are infinitely many $V_i$ if and only if $H$ is infinite if and only if $G$ is infinite; moreover, by density the $V_i$ are irreducible representations of $G$ any two which are inequivalent as $H$-representations are inequivalent as $G$-representations.
So to find a counterexample we should look for groups that are not residually finite. Infinite simple groups certainly have this property, but I don't know any of them well enough to analyze their irreducible representations.