This generalization is called a ternary commutator, see <A HREF="http://arxiv.org/abs/1207.6312">here.</A>