Suppose $X$ is a smooth family of algebraic varieties over the base $B:=\mathbb{P}^1\backslash\lbrace0,1,\infty\rbrace$ over $\overline{\mathbb{Q}}$; then we can form the relative $l$-adic cohomology on the base, which will be (having fixed some cohomological degree of interest) an $l$-adic sheaf; we can think of it as a $\mathbb{Q}_l$ vector space $V_l$ equipped with a $\pi_1(B)$ action. (Algebraic $\pi_1$, of course.)
We can do the same with $l'$-adic cohomology for some other prime $l'$, getting a vector space $V_{l'}$ over $\mathbb{Q}_{l'}$ equipped with a $\pi_1(B)$ action.
As far as I can see, there ought to be a relatively tight connection between these two groups. For instance, it seems morally like we ought to be able to choose pro-generators $\alpha,\beta$ for $\pi_1(B)$, and find some matrices $M_\alpha,M_\beta$ over $\overline{\mathbb{Q}}$, such that, w.r.t suitable bases of $V_l,V_{l'}$, $\alpha$ acts via $M_\alpha$ on both $V_l$ and $V_{l'}$ (thinking of $M_\alpha$ as determining an $l$- and $l'$- adic matrix in turn) and similarly for $M_\beta$. Can this indeed be done? If not, can you do something close? If it can, what's a good reference for a precisely stated theorem---rather than just intuition---for these things?