I also unfortunately don't really understand why people care so much about them, although I should probably go back and read old TWFs.
Sort of a combinatorial example of the fundamental groupoid is the category assigned to a graph where the objects are vertices and the morphisms are directed paths, and v->w->v = id_v. (If this makes sense.) I believe that this category is nice because (IIRC) you can read off the definition of graph homomorphism from it.
ETA: Okay, a quick Baez-review gives the following, which isn't strictly speaking what you asked about but which helps me understand how groupoids are intrinsically special and not just occasionally an improvement that encodes more data than groups.
If you move from categories to n-categories, you can define n-groupoids, although this is subtle. Now, just as the fundamental groupoid is a more natural construction than the fundamental group, n-groupoids capture more information about homotopy than do "monoidal n-groupoids." But furthermore, all the constructions of homotopy are in some way reversible (if I'm following Baez correctly) -- not only is homotopy equivalence an equivalence relation, but even on higher levels this is true -- e.g., the fundamental groupoid functor has an adjoint, which is essentially the classifying space construction, so actually n-categories capture everything about homotopy! And in fact, Baez says that if you think about \omega-categories (which are a limit of n-categories, essentially, I guess?) then the homotopy category of \omega-groupoids is equivalent to the homotopy category.