This is false for the cyclic group of order $6$. Let $g$ be a generator. Then $g^2-g+1$ acts by $0$ on the representations where $g$ acts by a primitive $6$-th root of $1$, and hence is not a unit in the group ring, but $1-1+1=1$ is relatively prime to $6$.

More generally, the statement is false whenever $G$ has an abelian quotient not of prime power order. Let $p$ and $q$ be two different primes dividing $G^{\mathrm{ab}}$, so there is a surjection $G \to \mathbb{Z}/(pq)$. Let $\chi$ be the character of $G$ formed by composing $G \to \mathbb{Z}/(pq)$ with an injection $\mathbb{Z}/(pq) \to \mathbb{C}^{\ast}$. Let $g \in G$ be such that $\chi(g)$ is a primitive $pq$-th root of unity, $\omega$.

Let $\Phi_{pq}(x) = \sum c_k x^k$ be the $pq$-th cyclotomic polynomial. So the element $\sum c_k g^k$ in $\mathbb{Z}[G]$ acts by $\sum c_k \chi(g)^k = \Phi_{pq}(\omega) =0$ on the representation $\chi$. Thus $\sum c_k g^k$ is not a unit. On the other hand, $\sum c_k = \Phi_{pq}(1)= 1$. (To compute the last, note that $\Phi_{pq}(x) = \frac{(x^{pq}-1)(x-1)}{(x^q-1)(x^p-1)}$ and take the limit as $x \to 1$.)

On the positive side, the statement is true whenever $G$ is a $p$-group. Let $\alpha = \sum c_g g \in \mathbb{Z}[G]$. I will show that the determinant of $\alpha$ acting on $\mathbb{Z}[G]$ is $\left( \sum c_g \right)^{|G|} \bmod p$, and hence is not $0$ if $\sum c_g \not \equiv 0 \bmod p$. Reducing $\mathbb{Z}[G]$ modulo $p$, we get an action of $\alpha$ on $\mathbb{F}_p[G]$. More generally, I claim that $\alpha$ acts on any $G$-representation $V$ over $\mathbb{F}_p$ by $\left( \sum c_g \right)^{\dim V}$. This is simple: $V$ has a filtration whose associated graded is a $\dim V$-dimensional trivial representation. Passing to the associated graded doesn't change determinant, and $\alpha$ acts on the $1$-dimensional trivial representation by $\sum c_g$.