Suppose that one has a Shimura variety $Sh(G,X)$ where $(G,X)$ is the corresponding Shimura datum and suppose that it can be interpreted as a moduli space of motives (e.g. PEL type Shimura variety, but also Hodge type and abelian type Shimura varieties). What is the relation between $G$ and the motivic Galois group (?) of the category of motives parametrized by $Sh(G,X)$?
(References for the exact definition of motivic Galois group being used in the answer would also be appreciated).