Words in two infinitismal rotations

Question

I asked this as subquestion in a comment pursuant to my Banach-Tarski question. I think it is worth promoting here to a question in its own right.

Consider these two matrices over ${\Bbb R}[[\epsilon]]$: $$A = \left[ \begin{array}{ccc} \cos(\epsilon) & \sin(\epsilon) & 0 \\ -\sin(\epsilon) & \cos(\epsilon) & 0 \\ 0 & 0 & 1 \end{array} \right] \ {\rm and}\ B = \left[ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & \cos(\epsilon) & \sin(\epsilon) \\ 0 & -\sin(\epsilon) & \cos(\epsilon) \end{array} \right].$$

(By $\sin$ and $\cos$ I mean the formal Taylor series.)

If a word $w$ in $A$ and $B$ equals the identity modulo $\epsilon^k$, must $w$ belong to the $k$th term of the derived series of the free group on the symbols $A$ and $B$?

Answer 1

This question is related (loosely) to the theory of discrete groups generated by small elements, which was developed and used for various purposes by Margulis and others. If $A$ and $B$ are elements of a Lie group, then in exponential coordinates around the identity, multiplication looks like addition to first order, and the commutator $A*B*A^{-1}*B^{-1}$ vanishes to first order, and looks like Lie bracket to second order: the errors are $o(|A||B|)$.

The Lie algebra of $SO(2)$ has a vector space basis $A, B, C$ where the Lie brackets are cross product, $[A,B]=C, [B,C]=A$ and $[C,A]=B$. In this Lie algebra, there is a commutator relation $[A,[B,[A,B]]] = 0$. This implies that the corresponding group word applied to the two matrices $A$ and $B$ above is $o(\epsilon^4)$, but it is only in the third term of the derived series for the free group. The mismatch will grow with more complicated commutators: $so(3)$ is just not big enough to accomodate the larger and larger dimension of the free nilpotent Lie algebras corresponding to the quotients of the free group by the terms in its lower central series. No finite dimensional Lie algebra is big enough.