Given an $l$-adic Galois representation which is geometric in the sense of Fontaine-Mazur what can one say about the set of (isomorphism classes of) of varieties whose $l$-adic cohomologies the representation occurs in, apart from the well-known conditions on their Hodge numbers and weights? Does it make sense to pose it as a moduli problem? How "large" is the set expected to be?

Thanks!

anyvariety $Y$ so the set in question isvery large. – Joël Sep 29 '11 at 17:04