The answer to the question might be yes (over an algebraically closed field of any characteristic), though it's not clearly documented in the literature.   First let me add a reference to the theorem of G.D. Mostow used by aakumadula: the paper *Self-adjoint groups*, Ann. of Math. 62 (1955), 44-55.   This is formulated for a "fully reducible" algebraic matrix group $G$ over $\mathbb{R}$ or $\mathbb{C}$, and asserts that a positive definite hermitian form exists relative to which $G$ is self-adjoint.   Thus after a change of basis, $G$ is closed under taking complex conjugate transposes.   The converse is easy, but the theorem itself relies heavily on the structure theory of relevant Lie groups and would require some work to adapt to arbitrary algebraically closed fields of characteristic 0.

The label *reductive* applies to the given complex algebraic group $G$ here.   By the general definition, a reductive algebraic group has a trivial unipotent radical but need not be connected.   In characteristic 0, it follows from Weyl's theorem (indirectly) that a reductive group acts in a completely/fully reducible way in any finite dimensional representation; this is familiar for finite groups.  But in prime characteristic this property fails badly for reductive groups including finite groups (except tori), so one has to rely on the full Borel-Chevalley structure theory.  This shows that a (say connected) reductive group is an almost-direct product of a torus and a semisimple (connected) algebraic group.   

With this in mind, I suspect the answer to the question is yes, though that isn't obvious or well documented in prime characteristic.   The crucial case is that of a connected reductive group (call it $H$) which is a subgroup of a reductive group $G$.   Write $N$ and $C$ respectively for the normalizer and centralizer of $H$ in $G$.   Then $C, H$ are normal subgroups of $N$, while their intersection is central in $H$. 

A standard consequence of the isomorphism theory for semisimple groups (Chevalley) is the theorem that inner automorphisms of such a group form a subgroup of finite index in the full automorphism group.   This is clearly needed to study $N$ when $H$ is semisimple.  In particular, $C \cap H$ is finite and $N$ is the almost-direct product of $C$ and $H$; this makes it most crucial to understand whether $C$ is reductive.  

A couple of special cases indicate the problem one still faces in prime characteristic.   If $H$ is a *torus*, an elementary result (rigidity) shows that its centralizer has finite index in its normalizer.  From the detailed Borel-Chevalley theory one eventually concludes that the centralizer is reductive (as well as connected).   So in this case $N$ is certainly reductive (though perhaps not connected).   

On the other hand, suppose $H$ is semisimple of rank 1, such as $\mathrm{SL}_2$.  Here the centralizer in $G$ is studied as part of the Bala-Carter parametrization of unipotent classes, but at first under a strong lower bound on the characteristic of the field which for example ensures that the Killing form of the Lie algebra of $G$ (when semisimple) is nondegenerate.  Here one finds via the Lie algebra that $C$ is indeed reductive.   But smaller primes require much more delicate study.

Nothing here is completely straightforward, so I wonder what the motivation for the question is?   This doesn't come up immediately in the basic structure theory.