It is true if C$C$ is the set of involutions $(i_1j_1)(i_2j_2)\cdots$ or even the set of all involutions with the added condition
$$i_1 \lt i_2 \lt \cdots\text{ and }j_1 \lt j_2 \lt \cdots \tag{$*$} $$
If a series is conditionally convergent, then its positive terms form a divergent series as do its negative terms. One can choose an involution which permutes the series to have runs of non-negative terms, each with sum greater than $1$ alternating with runs of negative terms each with sum less than $-1$.
This seems a little harder than I first thought so let me sketch some details. I'll build the involution a few transpositions at a time so at any stage there will be a finite partial permutation which fixes some terms and swaps some pairs of terms. The remaining terms will be termed available.
First find enough non-negative terms (in the given order) to add to a sum greater than $1$ and leave them as fixed points. If there are $j$ intervening negative terms, swap them with the next $j$ non-negative terms. These $j$ negative terms maintain their order. Any terms interspersed with these are negative as well and are left as fixed points. Now identify enough consecutive available negative terms, starting with the first such, to have a sum less than $-1$. If there are $j'$ intervening non-negative terms, swap them with the next $j'$ negative terms. Continue in this fashion.
Note that the involution constructed satisfies the added condition $(*)$.
There seems no harm in throwing in fixed-point free, at least if $(*)$ is dropped. Just swap previously fixed points of the same parity. With a litle more care one could probably have both $(*)$ and fixed-point free.