$hgh^{-1}=kgk^{-1}$ if and only if $k^{-1}h$ commutes with $g$.  So your first question is just asking for an infinite abelian subgroup $H$ that is its own centraliser in $G$.  There are many examples.  For instance, if $G$ is a non-abelian free group and $g\in G$ is not a proper power then $\langle g\rangle$ is its own centraliser.  Furthermore, normal subgroups of $G$ are either finite-index or infinitely generated, so $G$ is certainly not normal.