Let $X$ be a curve or an abelian variety (over a finite field). Then the Galois representation associated to $X$ via the etale cohomology of $X$ (in degree $1$) is integral of weight $1$ and its dimension is determined by the Hilbert polynomial of $X$. This is a theorem of Weil.
Let $X$ be a variety with fixed Hilbert polynomial $h$. Are the dimension and weight of the Galois representations associated to $X$ via etale cohomology determined by $h$? Note that these representations have a well-defined dimension and weight by Deligne's proof of the Riemann hypothesis over finite fields.
Edit: Will Sawin points out that the dimension of the representation doesn't only depend on the Hilbert polynomial. Thus, I would like to ask the following weaker question.
Is the dimension of the representation bounded if we fix the Hilbert polynomial?