Moduli spaces of arithmetic varieties with isomorphic $l$-adic cohomology

Given a positive integer $$d$$, a rational prime $$l$$ and a number field $$K$$, is it sensible to consider the moduli stack of $$d$$-dimensional varieties over $$K$$ whose $$l$$-adic cohomology rings are isomorphic as Galois modules?