The noncommutative space defined by $C[G]$ is (by definition) the dual $\widehat{G}$ of G. There are as many ways to make sense of this space as there are theories of noncommutative geometry.
(Edit: In particular if G is not finite you have lots of possible meanings for the group ring, depending on what kind of regularity and support conditions you put on G, or equivalently, what class of representations of G you wish to consider, and I will ignore all such issues - which are the main technical part of the subject - below.)
One basic principle is that noncommutative geometry is not about algebras up to isomorphism, it's about algebras up to Morita equivalence -- in other words, it's in fact about categories of modules over algebras (the basic invariant of Morita equivalence). You can think of these (depending on context) as vector bundles or sheaves of some kind on the dual. In this case we're looking at the category of complex representations of G, which are sheaves on the dual $\widehat{G}$. You can think of this as a form of the Fourier transform (modules for functions on G with convolution = modules for functions on the dual with multiplication), though obviously at this level of detail it's a complete tautology. Coarser invariants such as K-theory of group algebras, Hochschild homology etc give invariants, eg K-theory and cohomology, of the dual, noncommutative as it may be. There are many conjectures about this noncommutative topology, most famously the Baum-Connes conjecture relating the K-theory of this "space" to that of classifying spaces associated to G.
As to what the dual looks like, this is of course highly dependent on the group. For completely arbitrary groups I don't know of anything meaningful to say beyond structural things of the Baum-Connes flavor, so you have to pick a class of groups to study.
If G is abelian, the dual is itself a group (the dual group). The formal thing you can say in general is that the dual of G fibers over the dual of the center of G -- this is a form of Schur's lemma, saying irreducible reps live over a particular point of the dual of the center (ie the center acts by evaluation by a character). You might get some more traction by looking at the "Bernstein center" or Hochschild cohomology --- endomorphisms of the identity functor of G-reps. This is a commutative algebra and the dual fibers over its spectrum. In many cases this is a very good approximation to the dual -- ie the "fibers are finite" (this is what happens for say real and p-adic groups).
The orbit method of Kirillov says that for a nilpotent or solvable group, the dual looks like the dual space of the Lie algebra, modulo the coadjoint action. So again that's quite nice.
Very very roughly the Langlands philosophy says that for reductive groups G (in particular over local or finite fields) the dual of G is related to conjugacy classes in a dual group $G^\vee$. This is if you'd like a way to make meaningful the observation that conjugacy classes and irreps are in bijection for a finite group -- you roughly want to say they're in CANONICAL bijection if the two groups are "dual".
Rather than say it this coarsely, it's better to think in terms of the Harish-Chandra / Gelfand philosophy, which (again whittled down to one coarse snippet) says that the dual of a reductive group (over any field) is a union of "series", ie a union of subspaces each of which looks like the dual of a torus modulo a Weyl group. In other words, you look at all conjugacy classes of tori in G, for each torus you construct its dual (which is a group now!), and mod out by the symmetries inherited by the torus from its embedding in G, and this is the dual of G roughly. (This is also very close to saying semisimple conjugacy classes in the dual group of G, which is where the Langlands interpretation comes from). Anyway this is saying that the dual is a very nice and manageable, even algebraic, object. Kazhdan formulated this philosophy as saying that the dual of a reductive group is an algebraic object --- the reps of the group over a field F are something like the F-points of one fixed variety (or stack) over the algebraic closure. Anyway one can go much further, and that's what the Langlands program does.
