Now that you've added the maximality assumption, $\langle G,h\rangle$ has to be the normaliser of $G$ in $\mathop{\rm O}_n(\mathbb{R})$, and so does $\langle G,h'\rangle$. So they are not just conjugate, but equal. The set $H$ is the other coset of $G$ in its normaliser.