Thanks to Mark Wildon and the OP for pointing out that my answer was incorrect- I was considering the centralizer of $\sigma$ in the wrong algebra. However, it does seem to me that the structure of $A = C_{\mathbb{C}S_{N}}(\sigma)$ may dependdepends on the prime factorization of $N.$
In general it is well-known that if $\lambda$ is a partition of $N$ and $\chi_{\lambda}$ is the associated complex irreducible character of $S_{N}$, then $\chi_{\lambda}(\sigma) \in \{0,1,-1 \}$.
When $N= p$ is prime, this implies that ${\rm Res}^{S_{N}}_{\langle \sigma \rangle}(\chi_{\lambda})$ has one of the formsform $t\rho, t\rho + 1, t\rho -1$$t\rho + \chi_{\lambda}(\sigma)1, $ where $t$$t= \frac{\chi_{\lambda}(1) - \chi_{\lambda}(\sigma)}{p}$ is a non-negative integer and $\rho$ is the regular character of $\sigma$. But this inference can not be drawnis never the case for all $\chi_{\lambda}$ when $N$ is not prime, and is indeed false in general (for example, when $N = 4$ and $\chi_{\lambda}$ has a Klein $4$-subgroup in its kernel).
Now $\mathbb{C}S_{N}$ is isomorphic to $\bigoplus_{\lambda} M_{\chi_{\lambda}(1)}( \mathbb{C})$ as $\lambda$ runs through partitions of $N$. Now $\sigma$ acts as a matrix of trace $0$ or $\pm 1$ inside $M_{\chi_{\lambda}(1)}( \mathbb{C}).$
In the former case, the fixed subalgebra of $\sigma$ on the matrix algebra $M_{\chi_{\lambda}(1)}( \mathbb{C})$ has dimension $\frac{\chi_{\lambda}(1)^{2}}{p}.$
In the latter cases, we may compute the dimension of the fixed fixed subalgabra of $\sigma$ in the relevant matrix algebra.
If ${\rm Res}^{S_{N}}_{\langle \sigma \rangle }(\chi_{\lambda})= t_{\lambda} \rho \pm 1,$ then the fixed subalgebra of $\sigma$ in the matrix algebra has dimension $(p-1)t_{\lambda}^{2} + (t_{\lambda} \pm 1)^{2} = pt_{\lambda}^{2} \pm 2t_{\lambda} +1$.
UnfinishedThis means that when ....$N = p$ is prime, the dimension of the centralizer algebra of $\sigma$ in $\mathbb{C}S_{N}$ is totally detrmined by the values of the $\chi_{\lambda}(1)$ and $\chi_{\lambda}(\sigma)$, but his is not the case when $N$ is not prime.