Let $G$ be a group and let $\phi:G\to Z_2$ be a homomorphism to the group with two elements. Is it always the case that there are more conjugacy classes in the kernel of $\phi$ than conjugacy classes not in the kernel of $\phi$?

I've tried a little bit of messing around algebraically and written down some exact sequences of $G$-modules to try to apply the methods of group cohomology, but I haven't gotten anything to work.

My inspiration here is the special case when $G$ is the symmetric group $S_n$ and $\phi$ is the sign homomorphism. In this case conjugacy classes of $G$ correspond to partitions, and the problem becomes about counting partitions of $n$ with an even number of even parts versus an odd number of even parts. I was able to prove (via generating functions and also bijectively) that the number of partitions of $n$ with an even number of even parts minus the number of partitions of $n$ with an odd number of even parts is equal to the number of partitions of $n$ with all parts odd and distinct. I could not find a reference for this fact after some googling, so I would be interested to know if this is a well-known partition identity.

I'm also interested in possible extensions of this problem where $Z_2$ is replaced by another group $H$ (possibly required to be abelian).

(I just posted this on stackexchange and then learned that mathoverflow might be more appropriate but I'm not sure.)