The answer is affirmative for $G$ any smooth affine $k$-group. We may assume $p = {\rm{char}}(k) > 0$. By 14.26 in Borel's textbook on algebraic groups, $n$ is in the Lie algebra of a unipotent subgroup $U$ of $G$. The $p$-operator $Y \mapsto Y^{[p]}$ on the Lie algebra of a smooth affine group (or more generally group scheme of finite type) over $k$ is functorial in the group, and it vanishes for the Lie algebra of $\mathbf{G}_{\rm{a}}$, so via a suitable composition series on $U$ we see that $Y \mapsto Y^{[p^m]}$ kills ${\rm{Lie}}(U)$ for large $m$, and hence kills the nilpotent $n$. But since $[s,n]=0$, we have $X^{[p^i]} = s^{[p^i]} + n^{[p^i]}$ for all $i$, so $X^{[p^m]} = s^{[p^m]}$.
The theory of the $p$-operation works on Lie algebras of arbitrary group schemes over arbitrary $\mathbf{F}_p$-algebras (it is defined as the $p$-power on left-invariant derivations on the structure sheaf linear over the base ring), so by reasoning with Yoneda's Lemma we see that as closed subgroup schemes of $G$ we have
$$C_G(X) \subset C_G(X^{[p^m]}) = C_G(s^{[p^m]}).$$
Thus, to conclude that $C_G(X) \subset C_G(s)$ it suffices to show that $C_G(s) = C_G(s^{[p]})$
for any semisimple $s \in \mathfrak{g}$. Using an inclusion $G \hookrightarrow {\rm{GL}}_N$ reduces this to the case $G = {\rm{GL}}_N$ and diagonal $s \in \mathfrak{gl}_N = {\rm{Mat}}_N(k)$. Consideration of behavior with respect to weight-space decomposition then does the job since distinct elements of $k$ have distinct $p$th powers.
