Your conjecture is correct; as a matter of fact, it is possible to compute these sums $C(\mu):=\sum_\nu\chi_\nu(\mu)$ exactly for any $\mu$:
Suppose $\mu$ has $m_i$ parts of length $i$, for $i=1,2,\ldots$. Then $C(\mu)=\prod_{i>0} c_{i,m_i}$, where $c_{i,m_i}$ is the coefficient of $t^{m_i}/({m_i}!)$ in $\exp(t+\frac{1}{2}it^2)$ if $i$ is odd, or in $\exp(\frac{1}{2}it^2)$ if $i$ is even.
A first proof can be found in Macdonald's Symmetric functions and orthogonal Hall polynomials, ex.11 p.122; it relies on symmetric function techniques. A second one is given in Stanley's Enumerative Combinatorics Vol. 2, ex. 7.69, and is based on the fact that, from a general character theory result, $C(\mu)$ is the number of square roots of a given permutation of cycle-type $\mu$.

