If I'm not mistaken (but I often am), the physicists already have a rather simple way of defining the center of mass. But I don't think you can do it with just sets. You have to associate a mass with each set. The critical axiom is simply the one we all know:

If $A$ and $B$ are disjoint sets with masses $m(A)$ and $m(B)$ and center of masses $c(A)$ and $c(B)$ respectively, then the mass of $C$ is $m(C) = m(A) + m(B)$ and the center of mass of the set $C = A \cup B$ is given by
$$
c(C) = \frac{m(A)}{m(C)}c(A) + \frac{m(B)}{m(C)}c(B).
$$

You do need one more axiom to get started somehow. I believe physicists like to start with point masses (where the definition of the center of mass is easy) and then view a body as a limit of point masses. That's more or less what Liviu has proposed. But it also suffices to say that the center of mass of a square or cube is its geometric center. Or, more generally, the center of mass of any set with sufficient symmetry is its center.

Of course, if you really want arbitrary shapes, then you do need a countable version of the first axiom. But I think that's all you need. Note that this approach allows for bodies with different and even non-constant mass densities.

ADDED (in response to fedja's edit): It's worth noting explicitly that my answer above requires no notion of volume or choice of measure (such as Lebesgue measure) on the ambient space. It works on any length space.

But I don't see any way reduce this to just geometry (and not physics) without a notion of volume. In essence, you do it by just assuming all objects have the same constant mass density, so the mass is essentially equal to volume.

More generally, there has to be a way to measure the relative size of two sets, in order to determine the center of mass of the union of the two sets. If I understand correctly, axiom (4) in the question is an attempt to set this up. But isn't it simpler to define a notion of volume first and define the center of mass?

Valuations play a fundamental role in affine convex geometry (see, for example, the work of Monika Ludwig). Here, a valuation is essentially just a finitely additive measure. It seems to me that given any valuation or measure $m$ on the ambient space can be used to define a corresponding notion of center of mass $c$ as I've described above.

But this leads to an observation that maybe is what fedja is trying to tell us: It appears to me that if the ambient space is flat affine and $m$ is assumed to have reasonable properties, including an analogue of axiom (3) (such as $m(TE) = (\det T)^pm(E)$), then you always get the same center of mass, no matter what $m$ is.

So there should be a way to define the center of mass for sets in a flat affine space without using *any* valuation or measure at all.