While the comments by Angelo and Wilberd go a long way toward an answer, I'd prefer to start with a more precise formulation of the question. The group $G$ and its (closed!) subgroup $H$ may be assumed to be *connected* with $H$ moreover *reductive*. The structure theory involved here is essentially independent of the characteristic of the ground field, but on the other hand a field of definition apparently plays no role and could be assumed to be algebraically closed.

Up to isogeny, a connected reductive group is a direct product of closed connected subgroups which can be tori or products of (quasi-)simple groups of a fixed type. Due to the standard theorem on "rigidity" of tori, the
centralizer of a torus in $G$ has finite index in the normalizer. On the other hand, the outer automorphism group of a simple group is finite (coming from the automorphisms of the Dynkin diagram) while other outer automorphisms of a direct product of groups of the same type come from just finitely many permutations of isomorphic simple factors.

Since an automorphism of $H$ preserves the center and the various products of simple factors of the same type (whose overall product is the derived group of $H$), these pieces combine to show that $N_G(H)/H \, Z_G(H)$ is finite. (Though I haven't written it all down myself.)
I don't recall a specific reference for such a general result in the literature; it may only come up in special situations.