In my opinion the richness of the structure is related to endomorphisms of such set. It i s note necessary the group of symmetry but definitely it has something in common to possible relations within elements of such set. Even if You cannot have symmetry within set, it may have rich structure, for example even if there is no inverse elements or identity, there may be many interesting properties.
Remarks:
If You agree with above, it probably we may go further and try to define the measure for this "richness". For example You may try to construct some functors from given class of endomorphisms over structure to some predefined and known categories for example. Maybe there is possible to find something analogous to class of homotopy etc. in such construction? Probably it would be very interesting thing to define "richness invariants" which would be some constructions staying at some level whilst between similarly rich structures.
Note that it is not that the number ( cardinality) of such endomorphism decide about richness, but rather its "strangeness". For example class of endomorphisms of general function between two complex planes is much bigger that class of conformal mapping of complex planes, but the last class is definitely more interesting than first one. The last one is too big so not very interesting...