There is a strange product that takes two square roots of unit matrix, say $A$ and $B$, $A^2=I$, $B^2=I$ to a square root again, $$ A\star B=(A+B)^{-1}(A-B+2I), \qquad (A\star B)^2=I$$ Could anybody help me with identifying this structure? Where it comes from?

It was obtained from the Caley transform $C(A)=(1-A)^{-1}(1+A)$, $C(C(A))=A$, by expanding $C(C(A),C(B))$ and using $A^2=B^2=I$. Ones we imposed $A^2=B^2=I$ the inverse transform does not exists anymore, so we are at the singular point of the Caley transform. Somehow it looks like adding some points at infinity and extending the action on them.