The question is equivalent to asking whether $\mathbb{C}G$ contains non-zero nilpotent elements. As Yemon Choi's answer implicitly demonstrates, the answer is yes if and only if $G$ is non-Abelian.

This is a consequence of the Wedderburn structure theorems applied to the semisimple algebra $\mathbb{C}G$, as has been noted in comments.

Here is an elementary proof that the group algebra $\mathbb{C}G$ contains no non-zero nilpotent right ideal, which is equivalent to the semisimplicity of $\mathbb{C}G$ but the proof ( which has appeared in the book on character theory by D. Goldschmidt) requires little machinery.

Recall that a right ideal $I$ is nilpotent if every sufficiently long product of elements of $I$ is zero. In particular, every element of $I$ must be nilpotent.

Suppose that $I$ is a nilpotent right ideal of $\mathbb{C}G$ and that $r \in I$. Then $rg$ is nilpotent for each $g \in G$.

Consider the right regular matrix representation of $\mathbb{C}G$, and let $t$ be the trace it affords. Then $t(1_{G}) = |G|$ and $t(g) = 0 $ for all $g \neq 1_{G} \in G$.

Hence for every $g \in G$, we have that $t(rg^{-1})$ is $|G|$ times the coefficient of $g$ in $r$ ( when $r$ is expressed as a linear combination of elements of $G$).

However $rg^{-1}$ is nilpotent for all $g \in G$ as $I$ is a right ideal, and nilpotent matrices always have trace zero, so $t(rg^{-1}) = 0$ for all $g \in G$, and thus $r = 0$.