Interesting finite groups tend to have interesting inherent geometries (just as orbit-stabilizer turns external actions into internal actions, similar ideas turn many external geometries into coset geometries). The geometry induced by conjugation on Sylow p-subgroups is important for all finite groups, and turns out to describe the (p-completion of the) classifying space of the group.
Geometry has always been an important part of group theory. Zassenhaus groups and sharply triply transitive groups typically have an underlying affine or projective plane they are acting on. Early investigations of these special permutation groups in the 1930s led to some of the systematic development of finite geometry over things other than fields. You can recover the algebraic structure of something like a ring just from the permutation action of the group (often on a regular subgroup). M. Hall Jr.'s textbook on the Theory of Groups has a nice exposition of these ideas.
Of course finite groups of Lie type acting on their Borel subgroups also define important geometries, roughly called "buildings", and there are a great many references for those. This became a very popular way to understand the non-sporadic groups. These groups of Lie type have other nice actions, often on interesting finite geometries called generalized polygons.
Equivariant homotopy people noticed that some of these geometries are nearly enough to define a classifying space of the group, along with a nice decomposition of its cohomology ring. D. Benson and S.D. Smith's book on Classifying Spaces of Sporadic Simple Groups (MR2378355) describes these techniques with a reasonably algebraic feel. Modulo a few details, these are the fusion systems Scott Carnahan mention in a previous thread, MO5659. These geometries were investigated in order to provide a more natural analogue of buildings for sporadic groups.
Actually, I suppose you might feel that classifying spaces themselves are naturally associated to finite groups.
Edit: I thought it might be helpful to point out the similarities to the Zariski topology: The Zariski topology basically encodes how prime ideals intersect. The fusion of a finite group encodes how Sylow subgroups intersect. Strong fusion not only keeps track of the intersections, but also of the (G-inner) maps between those intersections, so that the fusion becomes a category. Since fusion controls cohomology, it seemed natural to look at how fusion describes the classifying space of a group. Amazingly, it does a great job of describing the p-completion of the classifying space and facilitates fairly direct calculations. In other words, the data encoded by the "prime subgroups" (Sylow p-subgroups) also encodes a natural topological space associated to the group, its (p-completed) classifying space.
Several areas of combinatorics, like certain parts of graph theory and finite geometry, also seem to be based on the simple fact that interesting groups have interesting geometries. A recent classification of Steiner triple systems followed from detailed classifications of finite simple groups and multiply transitive permutation groups, and several families of graphs are interesting because of their automorphism groups.
I hope it is clear too that separating a group from its actions is not sensible. The actions of a group are encoded by the conjugacy classes of its subgroups, and it is entirely internal. Most geometries associated to groups are also internal. This is basically why the classification of finite simple groups can succeed: the natural action of a group is already contained inside it in an easy to describe way, so that once the local structure of a group is sufficiently similar to a known group, the group itself is isomorphic to a known group.