This is an elaboration on ACL's answer, way too long for a comment, which highlights a technical ingredient (well-known to all experts) that underlies the precise sense in which the $\ell$-adic etale cohomology of the geometric generic fiber provides a "uniformity" in $p$: the good properties of constructible $\ell$-adic sheaves. In particular, I think it is a mistake to try to understand a precise sense of "uniformity in $p$" by focusing on point-counting: this misses the key structure, as noted in ACL's answer, namely certain $\ell$-adic representations (of the absolute Galois group of $\mathbf{Q}$) which individually are not expressed via point-counting at all.

Literally from the product definition, the zeta function of a separated finite type $\mathbf{Z}$-scheme $X$ is the product $\prod_p \zeta(X_{\mathbf{F}_p}, p^{-s})$ of the zeta functions of the fibers, with ${\rm{Re}}(s)$ is sufficiently large (determined by fiber dimensions alone; see Serre's article in the Purdue conference proceedings on arithmetic geometry from the mid-1960's). By the work of Dwork or Grothendieck-Deligne, the zeta function of any separated scheme of finite type over $\mathbf{F}_p$ (such as $X_{\mathbf{F}_p}$) is a rational function in $p^{-s}$.

The cohomological formalism provides an "$\ell$-adic" explanation for the rationality of the factor at each prime $p$ in the sense that for any prime $\ell \ne p$ we have $$\zeta(X_{\mathbf{F}_p}, t) = \prod_{i\ge 0} \det(1 - \phi_p t| {\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p}, \mathbf{Q}_{\ell}))^{(-1)^{i+1}}$$ in $\mathbf{Q}_{\ell}[\![t]\!]$, where the left side is initially just a formal power series in $1 + t \mathbf{Z}[\![t]\!]$ (defined as a product over closed points of $X_{\mathbf{F}_p}$) and the rational function over $\mathbf{Q}_{\ell}$ on the right side might involve internal cancellations among the various determinant polynomials (ruled out for smooth proper $X_{\mathbf{F}_p}$ by the Deligne's work on the Riemann Hypothesis, but not otherwise). In other words, the "$\ell$-adic" explanation for rationality rests on the fact that $\mathbf{Q}(\!(t)\!) \cap \mathbf{Q}_{\ell}(t) = \mathbf{Q}(t)$ inside $\mathbf{Q}_{\ell}(\!(t)\!)$ (and Dwork's approach provides a variant of that explanation with $\ell=p$). In the displayed product on the right side, $i$ goes up to $2 \dim X_{\mathbf{F}_p}$ (which is bounded independently of $p$, and in fact equal to $2 \dim X_{\mathbf{Q}}$ for all but finitely many $p$).

That was all just setup. Now fix a prime $\ell$ and an integer $i \ge 0$. One can ask if there is a finite set $S_{i,\ell}$ of primes of $\mathbf{Z}$ with $\ell \in S_{i,\ell}$ such that the polynomials $$R_{p,i,\ell}(t) = \det(1 - \phi_p t|{\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p},\mathbf{Q}_{\ell}))$$ for all $p \not\in S_{i,\ell}$ are "linked" in the sense that there is a single finite-dimensional continuous $\mathbf{Q}_{\ell}$-linear representation $$\rho_{i,\ell}: G_{\mathbf{Q},S_{i,\ell}} \rightarrow {\rm{GL}}(V_{i,\ell})$$ of the Galois group over $\mathbf{Q}$ of its maximal extension (inside $\overline{\mathbf{Q}}$) unramified outside $S_{i,\ell}$ such that $$\det(1 - t \rho_{i,\ell}(\phi_p)|V_{i,\ell}) = R_{p,i,\ell}(t)$$ for all $p \not\in S_{i,\ell}$, where $\phi_p \in G_{\mathbf{Q},S_{i,\ell}}$ is a member of the conjugacy class of geometric Frobenius elements at $p$ (all choices giving the same determinant). This would imply in particular that the degree of $R_{p,i,\ell}(t)$ is the same for all $p \not\in S_{i,\ell}$, but it is a much stronger statement: that $\rho_{i,\ell}$ would be a kind of "$\ell$-adic glue" which unifies the disparate $R_{p,i,\ell}(t)$'s coming from the geometric special fibers $X_{\overline{\mathbf{F}}_p}$ in varying characteristics $p \not\in S_{i,\ell}$.

The crux of the matter then is the following fundamental fact: the continuous representation $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}},\mathbf{Q}_{\ell})$ is such a $\rho_{i,\ell}$, for an appropriate choice of $S_{i,\ell}$. Why? Here is where one has to use a real theorem, namely the preservation of constructibility of $\ell$-adic sheaves under higher direct images with proper support, coupled with the proper base change theorem. More precisely, if $h:Y' \rightarrow Y$ is any separated map of finite type between noetherian schemes over $\mathbf{Z}[1/\ell]$ and if $\mathscr{F}$ is any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y'$ (e.g., the constant sheaf $\mathbf{Q}_{\ell}$) then ${\rm{R}}^i h_{!}(\mathscr{F})$ is a constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ whose formation moreover commutes with any base change (the latter due to the proper base change theorem). The point is that any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ is lisse over a dense open $U$ (depending on the sheaf), and hence "is" just a continuous $\mathbf{Q}_{\ell}$-linear representation of the fundamental group $\pi_1(U,\eta)$ if $Y$ is normal and connected (with geometric generic point $\eta$). In particular, when $Y$ is a connected Dedekind scheme then over $U$ this lisse sheaf is nothing more or less than an $\ell$-adic representation $\rho$ of the absolute Galois group of the function field of $Y$ (i.e., the residue field at the generic point of $Y$) such that $\rho$ is unramified at all closed points $u$ of $U$. The Galois representation at $u$ arising from the $u$-stalk of the lisse sheaf coincides with the residual Galois representation arising from $\rho$ on the Galois group at the generic point by virtue of its unramifiedness at $u$ (upon choosing a decomposition group at $u$ in the Galois group at the generic point, which amounts to working with a strict henselization at $u$ inside a separable closure of the function field of $Y$ in order to compute the specialization homomorphism from geometric stalk at $u$ to a geometric generic stalk).

For example, take $Y' = X_{\mathbf{Z}[1/\ell]}$ and $Y = {\rm{Spec}}(\mathbf{Z}[1/\ell])$ and $\mathscr{F} = \mathbf{Q}_{\ell}$. The above says that there is a dense open subscheme $U_{i,\ell} \subset {\rm{Spec}}(\mathbf{Z}[1/\ell])$ such that the constructible ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})$ on ${\rm{Spec}}(\mathbf{Z}[1/\ell])$ has restriction over $U_{i,\ell}$ that is lisse. Letting $S_{i,\ell}$ be the finite set of closed points of ${\rm{Spec}}(\mathbf{Z})$ complementary to $U_{i,\ell}$, we have that $\pi_1(U_{i,\ell}) = G_{\mathbf{Q},S_{i,\ell}}$ (using geometric generic point as base point of $\pi_1$) and the lisse restriction ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})|_{U_{i,\ell}}$ has respective stalks at the chosen geometric generic point and geometric closed point at $p \not\in S_{i,\ell}$ identified as Galois modules (for $\mathbf{Q}$ and $\mathbf{F}_p$ respectively) with the respective geometric fibral cohomologies $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}}, \mathbf{Q}_{\ell})$ and ${\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p},\mathbf{Q}_{\ell})$ (recovering in particular that $V_{i,\ell}$ is unramified at $p$, as we know it must be due to $V_{i,\ell}$ arising from a $\pi_1(U_{i,\ell})$-representation). In other words, it is precisely the lisse pullback of ${\rm{R}}^if_{!}(\mathbf{Q}_{\ell})$ over ${\rm{Spec}}(\mathbf{Z}_{(p)})$ viewed as a representation of $\pi_1({\rm{Spec}}(\mathbf{Z}_{(p)}))$ which is the "$\ell$-adic glue" that links up the $i$th factor in the $\ell$-adic alternating product formula for $\zeta(X_{\overline{\mathbf{F}}_p},t)$ with the single entity $V_{i,\ell}$ that "doesn't know $p$". And the mechanism of this linkage is that (up to conjugation ambiguity!) we can compute that $\pi_1$ using geometric base points over either the generic or closed points of ${\rm{Spec}}(\mathbf{Z}_{(p)})$.

Note in particular that each $V_{i,\ell}$ on its own does not have anything to do with point-counting. It is only the alternating product built from these which is related to point-counting. But it is the $V_{i,\ell}$'s which are where the action is.

This is an elaboration on ACL's answer, way too long for a comment, which highlights a technical ingredient (well-known to all experts) that underlies the precise sense in which the $\ell$-adic etale cohomology of the geometric generic fiber provides a "uniformity" in $p$: the good properties of constructible $\ell$-adic sheaves. In particular, I think it is a mistake to try to understand a precise sense of "uniformity in $p$" by focusing on point-counting: this misses the key structure, as noted in ACL's answer, namely certain $\ell$-adic representations (of the absolute Galois group of $\mathbf{Q}$) which individually are not expressed via point-counting at all (away from misleading special cases such as curves and abelian varieties for which degree-1 cohomology over finite fields contains all of the cohomological information).

Literally from the product definition, the zeta function of a separated finite type $\mathbf{Z}$-scheme $X$ is the product $\prod_p \zeta(X_{\mathbf{F}_p}, p^{-s})$ of the zeta functions of the fibers, with ${\rm{Re}}(s)$ is sufficiently large (determined by fiber dimensions alone; see Serre's article in the Purdue conference proceedings on arithmetic geometry from the mid-1960's). By the work of Dwork or Grothendieck-Artin (et al.), the zeta function of any separated scheme of finite type over $\mathbf{F}_p$ (such as $X_{\mathbf{F}_p}$) is a rational function in $p^{-s}$.

The cohomological formalism provides an "$\ell$-adic" explanation for the rationality of the factor at each prime $p$ in the sense that for any prime $\ell \ne p$ we have $$\zeta(X_{\mathbf{F}_p}, t) = \prod_{i\ge 0} \det(1 - t\phi_p| {\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p}, \mathbf{Q}_{\ell}))^{(-1)^{i+1}}$$ in $\mathbf{Q}_{\ell}[\![t]\!]$, where the left side is initially just a formal power series in $1 + t \mathbf{Z}[\![t]\!]$ (defined as a product over closed points of $X_{\mathbf{F}_p}$) and the rational function over $\mathbf{Q}_{\ell}$ on the right side might involve internal cancellations among the various determinant polynomials (ruled out for smooth proper $X_{\mathbf{F}_p}$ by the Deligne's work on the Riemann Hypothesis, but not otherwise). In other words, the "$\ell$-adic" explanation for rationality rests on the fact that $\mathbf{Q}(\!(t)\!) \cap \mathbf{Q}_{\ell}(t) = \mathbf{Q}(t)$ inside $\mathbf{Q}_{\ell}(\!(t)\!)$ (and Dwork's approach provides a variant of that explanation with $\ell=p$). In the displayed product on the right side, $i$ goes up to $2 \dim X_{\mathbf{F}_p}$ (which is bounded independently of $p$, and in fact equal to $2 \dim X_{\mathbf{Q}}$ for all but finitely many $p$).

That was all just setup. Now fix a prime $\ell$ and an integer $i \ge 0$. One can ask if there is a finite set $S_{i,\ell}$ of primes of $\mathbf{Z}$ with $\ell \in S_{i,\ell}$ such that the polynomials $$R_{p,i,\ell}(t) = \det(1 - t \phi_p|{\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p},\mathbf{Q}_{\ell}))$$ for all $p \not\in S_{i,\ell}$ are "linked" in the sense that there is a single finite-dimensional continuous $\mathbf{Q}_{\ell}$-linear representation $$\rho_{i,\ell}: G_{\mathbf{Q},S_{i,\ell}} \rightarrow {\rm{GL}}(V_{i,\ell})$$ of the Galois group over $\mathbf{Q}$ of its maximal extension (inside $\overline{\mathbf{Q}}$) unramified outside $S_{i,\ell}$ such that $$\det(1 - t \rho_{i,\ell}(\phi_p)|V_{i,\ell}) = R_{p,i,\ell}(t)$$ for all $p \not\in S_{i,\ell}$, where $\phi_p \in G_{\mathbf{Q},S_{i,\ell}}$ is a member of the conjugacy class of geometric Frobenius elements at $p$ (all choices giving the same determinant). This would imply in particular that the degree of $R_{p,i,\ell}(t)$ is the same for all $p \not\in S_{i,\ell}$, but it is a much stronger statement: that $\rho_{i,\ell}$ would be a kind of "$\ell$-adic glue" which unifies the disparate $R_{p,i,\ell}(t)$'s coming from the geometric special fibers $X_{\overline{\mathbf{F}}_p}$ in varying characteristics $p \not\in S_{i,\ell}$.

The crux of the matter then is the following fundamental fact: the continuous representation $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}},\mathbf{Q}_{\ell})$ is such a $\rho_{i,\ell}$, for an appropriate choice of $S_{i,\ell}$. Why? Here is where one has to use a real theorem, namely the preservation of constructibility of $\ell$-adic sheaves under higher direct images with proper support, coupled with the proper base change theorem. More precisely, if $h:Y' \rightarrow Y$ is any separated map of finite type between noetherian schemes over $\mathbf{Z}[1/\ell]$ and if $\mathscr{F}$ is any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y'$ (e.g., the constant sheaf $\mathbf{Q}_{\ell}$) then ${\rm{R}}^i h_{!}(\mathscr{F})$ is a constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ whose formation moreover commutes with any base change (the latter due to the proper base change theorem).

The point is that any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ is lisse over a dense open $U$ (depending on the sheaf), and hence "is" just a continuous $\mathbf{Q}_{\ell}$-linear representation of the fundamental group $\pi_1(U,\eta)$ if $Y$ is normal and connected (with geometric generic point $\eta$). In particular, when $Y$ is a connected Dedekind scheme then over $U$ this lisse sheaf is nothing more or less than an $\ell$-adic representation $\rho$ of the absolute Galois group of the function field of $Y$ (i.e., the residue field at the generic point of $Y$) such that $\rho$ is unramified at all closed points $u$ of $U$. The Galois representation at $u$ arising from the $u$-stalk of the lisse sheaf coincides with the residual Galois representation arising from $\rho$ on the Galois group at the generic point by virtue of its unramifiedness at $u$ (upon choosing a decomposition group at $u$ in the Galois group at the generic point, which amounts to working with a strict henselization at $u$ inside a separable closure of the function field of $Y$ in order to compute the specialization homomorphism from geometric stalk at $u$ to a geometric generic stalk).

For example, take $Y' = X_{\mathbf{Z}[1/\ell]}$ and $Y = {\rm{Spec}}(\mathbf{Z}[1/\ell])$ and $\mathscr{F} = \mathbf{Q}_{\ell}$. The above says that there is a dense open subscheme $U_{i,\ell} \subset {\rm{Spec}}(\mathbf{Z}[1/\ell])$ such that the constructible ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})$ on ${\rm{Spec}}(\mathbf{Z}[1/\ell])$ has restriction over $U_{i,\ell}$ that is lisse. Letting $S_{i,\ell}$ be the finite set of closed points of ${\rm{Spec}}(\mathbf{Z})$ complementary to $U_{i,\ell}$, we have that $\pi_1(U_{i,\ell}) = G_{\mathbf{Q},S_{i,\ell}}$ (using geometric generic point as base point of $\pi_1$) and the lisse restriction ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})|_{U_{i,\ell}}$ has respective stalks at the chosen geometric generic point and geometric closed point at $p \not\in S_{i,\ell}$ identified as Galois modules (for $\mathbf{Q}$ and $\mathbf{F}_p$ respectively) with the respective geometric fibral cohomologies $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}}, \mathbf{Q}_{\ell})$ and ${\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p},\mathbf{Q}_{\ell})$ (recovering in particular that $V_{i,\ell}$ is unramified at $p$, as we know it must be due to $V_{i,\ell}$ arising from a $\pi_1(U_{i,\ell})$-representation).

In other words, it is precisely the lisse pullback of ${\rm{R}}^if_{!}(\mathbf{Q}_{\ell})$ over ${\rm{Spec}}(\mathbf{Z}_{(p)})$ viewed as a representation of $\pi_1({\rm{Spec}}(\mathbf{Z}_{(p)}))$ which is the "$\ell$-adic glue" that links up the $i$th factor in the $\ell$-adic alternating product formula for $\zeta(X_{\overline{\mathbf{F}}_p},t)$ with the single entity $V_{i,\ell}$ that "doesn't know $p$". And the mechanism of this linkage is that (up to conjugation ambiguity!) we can compute that $\pi_1$ using geometric base points over either the generic or closed points of ${\rm{Spec}}(\mathbf{Z}_{(p)})$.

Note in particular that each $V_{i,\ell}$ on its own does not have anything to do with point-counting (away from special cases like curves and abelian varieties). It is only the alternating product built from these which is related to point-counting. But it is the $V_{i,\ell}$'s which are where the action is.

(If you look at the Introduction to deJong's IHES paper on alterations you'll see that he was initially hopeful that his results replacing absence of resolutions of singularities in positive characteristic might have applications to prove new "independence of $\ell$" results, but that this didn't pan out; I am not aware of anyone having made substantial progress on it since that time either, but would be happy to hear to the contrary. Even if we grant resolution of singularities then I don't think an implication is known. In the absence of precise control on weights as the purity provided by RH in the smooth proper case, it is hard to geometrically isolate the contribution in a single cohomological degree from the rest, as the long exact excision sequence associated to a stratification lumps together all cohomological degrees. Deligne's Weil II is very suggestive, but alas I think not enough even assuming resolution.)

The crux of the matter then is the following fundamental fact: the continuous representation $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}},\mathbf{Q}_{\ell})$ is such a $\rho_{i,\ell}$, for an appropriate choice of $S_{i,\ell}$. Why? Here is where one has to use a real theorem, namely the preservation of constructibility of $\ell$-adic sheaves under higher direct images with proper support, coupled with the proper base change theorem. More precisely, if $h:Y' \rightarrow Y$ is any separated map of finite type between noetherian schemes over $\mathbf{Z}[1/\ell]$ and if $\mathscr{F}$ is any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y'$ (e.g., the constant sheaf $\mathbf{Q}_{\ell}$) then ${\rm{R}}^i h_{!}(\mathscr{F})$ is a constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ whose formation moreover commutes with any base change (the latter due to the proper base change theorem). The point is that any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ is lisse over a dense open $U$ (depending on the sheaf), and hence "is" just a continuous $\mathbf{Q}_{\ell}$-linear representation of the fundamental group $\pi_1(U,\eta)$ if $Y$ is normal. In particular, when $Y$ is a connected Dedekind scheme then over $U$ this lisse sheaf is nothing more or less than an $\ell$-adic representation $\rho$ of the absolute Galois group of the function field of $Y$ (i.e., the residue field at the generic point $\eta$ of $Y$) such that $\rho$ is unramified at all closed points $u$ of $U$. The Galois representation at $u$ arising from the $u$-stalk of the lisse sheaf coincides with the residual Galois representation arising from $\rho$ on the Galois group at $\eta$ by virtue of its unramifiedness at $u$ (upon choosing a decomposition group at $u$ in the Galois group at $\eta$, which amounts to working with a strict henselization at $u$ inside a separable closure of $\eta$ in order to compute the specialization homomorphism from geometric stalk at $u$ to a geometric generic stalk).

The crux of the matter then is the following fundamental fact: the continuous representation $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}},\mathbf{Q}_{\ell})$ is such a $\rho_{i,\ell}$, for an appropriate choice of $S_{i,\ell}$. Why? Here is where one has to use a real theorem, namely the preservation of constructibility of $\ell$-adic sheaves under higher direct images with proper support, coupled with the proper base change theorem. More precisely, if $h:Y' \rightarrow Y$ is any separated map of finite type between noetherian schemes over $\mathbf{Z}[1/\ell]$ and if $\mathscr{F}$ is any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y'$ (e.g., the constant sheaf $\mathbf{Q}_{\ell}$) then ${\rm{R}}^i h_{!}(\mathscr{F})$ is a constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ whose formation moreover commutes with any base change (the latter due to the proper base change theorem). The point is that any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ is lisse over a dense open $U$ (depending on the sheaf), and hence "is" just a continuous $\mathbf{Q}_{\ell}$-linear representation of the fundamental group $\pi_1(U,\eta)$ if $Y$ is normal and connected (with geometric generic point $\eta$). In particular, when $Y$ is a connected Dedekind scheme then over $U$ this lisse sheaf is nothing more or less than an $\ell$-adic representation $\rho$ of the absolute Galois group of the function field of $Y$ (i.e., the residue field at the generic point of $Y$) such that $\rho$ is unramified at all closed points $u$ of $U$. The Galois representation at $u$ arising from the $u$-stalk of the lisse sheaf coincides with the residual Galois representation arising from $\rho$ on the Galois group at the generic point by virtue of its unramifiedness at $u$ (upon choosing a decomposition group at $u$ in the Galois group at the generic point, which amounts to working with a strict henselization at $u$ inside a separable closure of the function field of $Y$ in order to compute the specialization homomorphism from geometric stalk at $u$ to a geometric generic stalk).