I'd prefer to leave the question up still I think, since it really is the general behaviour for all $n$ that I'm interested in. However, I would accept any answer which can characterise the derivative using these kind of geometric considerations (for instance partial unitary invariance, as submultisets of $S^2$).

Ahh thanks, I added another condition, really I want to know if the derivative is uniquely defined by its metric behaviour on zeros (multisets), but I'm not sure how to best phrase that precisely.

Edited for clarity, I wasn't quite sure how to phrase the title, if you can think of better wording that would be very helpful.