The nicest way of phrasing it is the following. Let $\mathcal H$ be the category of Hilbert spaces with unitary maps between them. For each locally compact group $G$, one can define a functor
$$Rep_G : {\mathcal H} \to Top$$
with $Rep_G(H) = \hom(G,U(H))$, where the space of homomorphisms is endowed with the compact-open topology (with respect to the strong operator topology on $U(H)$). Obviously, the functor $Rep_G$ is compatible with sums and tensor products of Hilbert spaces. Note that 
$Rep_{\mathbb Z}(H)=U(H).$

Consider now any such functor $F: {\mathcal H} \to Top$ 
and set
$$D(F) = Nat_{\otimes,\oplus}(F,Rep_{\mathbb Z}),$$
i.e. all natural transformations of functors which are compatible with the tensor-product and the sum. $D(F)$ is a group since $Rep_{\mathbb Z}(H)=U(H)$ is a group for each Hilbert space $H$. It is also a topological group in a natural way.

Now, there is a natural map $\iota_G : G \to D(Rep_G)$ which is given by
$\iota(g)(\pi) = \pi(g)$, where $\pi \in hom(G,U(H))$. So just as in the case of Pontrjagin duality, there is a natural bi-dual. A bit of work (relying on results of Takesaki and Gel'fand-Raikov (which you have mentioned)) shows that $\iota_G$ is a topological isomorphism for all locally compact topological groups.

I studied the analogous question which arises if one restricts everything to the category of finite-dimensional Hilbert spaces (see [here][1]). This sometimes goes under the name Chu duality, but is not so extensively studied. Everything works for locally compact abelian groups and compact groups by Pontrjagins result and the Tannaka-Krein theorem. However, for finitely generated discrete groups, interesting things happen. First of all, it is trivial to observe that the analogous map

$$\iota_G : G \to D_{fin}(Rep_G)$$

is injective if and only $G$ is maximally almost periodic (by a result of Mal'cev iff $G$ is residually finite). Moreover, and this is more difficult, $\iota_G$ is an isomorphism if and only if $G$ is virtually abelian. In particular, for $G={\mathbb F_2}$, the map
$\iota_{\mathbb F_2}$ from $\mathbb F_2$ to $D_{fin}(Rep_{\mathbb F_2})$
is not surjective. This is a bit surprising as there are no natural candidates of elements in 
$D_{fin}(Rep_{\mathbb F_2})$, which do not lie in the image.

  [1]: http://arxiv.org/abs/1003.4093