It's not true as stated. You can take $\mu$ to be counting measure on
$H$, i.e. $\mu(A) = \#(A \cap H)$. This is trivially $H$-invariant
and $\sigma$-finite (the countably many points of $H$ each have finite
measure, and the rest of $G$ has measure zero), but it is not
$G$-invariant.
For a non-atomic example, take for instance $G = \mathbb{R}^2$, $H = \mathbb{Q}^2$, and $\mu$ the product of Lebesgue measure on $\mathbb{R}$ with counting measure on $\mathbb{Q}$, i.e. $\int f\,d\mu = \sum_{q \in \mathbb{Q}} \int_{\mathbb{R}} f(x, q)\,dx$.
However, if you suppose $\mu$ is Radon, then it's true. Let $g \in G$
be arbitrary and let $K \subset G$ be an arbitrary compact set. Let
$\epsilon > 0$, and let $h_n$ be a sequence in $H$ which converges to
$g$. Since $\mu$ is Radon we can find an open set $U \supset K$ such
that $\mu(U) \le \mu(K) + \epsilon$. Then for sufficiently large $n$,
we have $K + (g - h_n) \subset U$ as well. (If not, then passing to a subsequence we can find for each $n$ some $x_n \in K$ for which $x_n + g - h_n \notin U$. By compactness, passing to a further subsequence, $x_n$ converges to some $x \in K$. Then $x_n + g - h_n \to x$ which is absurd because $x \in K \subset U$.)
Thus $$\mu(K+g) =
\mu(K+(g-h_n)) \le \mu(U) \le \mu(K) + \epsilon.$$ Since $\epsilon$
was arbitrary we have $\mu(K+g) \le \mu(K)$, and the reverse
inequality follows by symmetry. Using again the fact that $\mu$ is
Radon, every Borel set can be approximated in measure from within by
compact sets, and so this suffices to conclude that $\mu$ is
$G$-invariant.
I think this is true, with much the same proof, for any topological group $G$ (not necessarily abelian or metrizable or locally compact) and any dense subgroup $H$ (not necessarily countable). If $G$ is not abelian then it says that if $\mu$ is invariant under left (respectively, right) multiplication by $H$, then $\mu$ is left (right) invariant. If $G$ should fail to be first countable you can replace the sequence $h_n$ in the above argument with a net.
