Actually, the category of motives isn't equivalent to the category of representations of an affine group scheme except in characteristic zero, and even there the equivalence depends on the choice of a fibre functor.

The functor to motives is supposed to be a universal cohomology theory. Certainly, one would like the target of a cohomology theory to be at least tannakian.

If you assume the Hodge conjecture, then the affine group scheme attached to the category of abelian motives over $\mathbb{C}$ (that generated by abelian varieties) is more-or-less known --- at least its algebraic quotients are classified.

If you assume the Tate conjecture, then the affine groupoid attached to the category of motives over an algebraic closure of $\mathbb{F}_p$ is more-or-less known.

In the general case nothing is known except that the group is VERY BIG --- for example, over $\mathbb{C}$ it has uncountably many distinct quotients isomorphic to PGL(2).