In this definition BQM can be taken to be a space - the geometric realization of the nerve of the category QM. The homotopy groups are then the usual homotopy groups from topology.
There also is a definition of the homotopy group of a simplicial set - you can thus compute the homotopy group of the nerve without passing to the geometric realization first - and the definition you give looks more like that, but the answers are isomorphic.