This is an elaboration on ACL's answer, way too long for a comment, which highlights a technical ingredient (well-known to all experts) that underlies the precise sense in which the $\ell$-adic etale cohomology of the geometric generic fiber provides a "uniformity" in $p$: the good properties of constructible $\ell$-adic sheaves.
In particular, I think it is a mistake to try to understand a precise sense of "uniformity in $p$" by focusing on point-counting: this misses the key structure, as noted in ACL's answer, namely certain $\ell$-adic representations (of the absolute Galois group of $\mathbf{Q}$) which individually are not expressed via point-counting at all (away from misleading special cases such as curves and abelian varieties for which degree-1 cohomology over finite fields contains all of the cohomological information).
To explain this requires some preparations, hence the length of what follows (which is all standard stuff, but perhaps hard to extract for a non-expert; maybe even what follows is hard to read in parts for a non-expert, but I think it is important to recognize where serious theorems of etale cohomology are doing some work, going beyond the cohomological formula for the zeta function of a single variety over a single finite field). The crux is that the robustness of constructibility provides the magical glue linking behavior at different primes.
Literally from the product definition, the zeta function of a separated finite type $\mathbf{Z}$-scheme $X$ is the product $\prod_p \zeta(X_{\mathbf{F}_p}, p^{-s})$ of the zeta functions of the fibers, with ${\rm{Re}}(s)$ is sufficiently large (determined by fiber dimensions alone; see Serre's article in the Purdue conference proceedings on arithmetic geometry from the mid-1960's). By the work of Dwork or Grothendieck-Artin (et al.), the zeta function of any separated scheme of finite type over $\mathbf{F}_p$ (such as $X_{\mathbf{F}_p}$) is a rational function in $p^{-s}$.
The cohomological formalism provides an "$\ell$-adic" explanation for the rationality of the factor at each prime $p$ in the sense that
for any prime $\ell \ne p$ we have
$$\zeta(X_{\mathbf{F}_p}, t) = \prod_{i\ge 0} \det(1 - t\phi_p|
{\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p}, \mathbf{Q}_{\ell}))^{(-1)^{i+1}}$$
in $\mathbf{Q}_{\ell}[\![t]\!]$, where the left side is initially just a formal power series in $1 + t \mathbf{Z}[\![t]\!]$ (defined as a product over closed points of $X_{\mathbf{F}_p}$) and the rational function over $\mathbf{Q}_{\ell}$ on the right side might involve internal cancellations among the various determinant polynomials (ruled out for smooth proper $X_{\mathbf{F}_p}$ by the Deligne's work on the Riemann Hypothesis, but not otherwise). In other words, the "$\ell$-adic" explanation for rationality rests on the fact that $\mathbf{Q}(\!(t)\!) \cap \mathbf{Q}_{\ell}(t) = \mathbf{Q}(t)$ inside $\mathbf{Q}_{\ell}(\!(t)\!)$ (and Dwork's approach provides a variant of that explanation with $\ell=p$). In the displayed product on the right side, $i$ goes up to $2 \dim X_{\mathbf{F}_p}$ (which is bounded independently of $p$, and in fact equal to $2 \dim X_{\mathbf{Q}}$ for all but finitely many $p$).
That was all just setup. Now fix a prime $\ell$ and an integer $i \ge 0$. One can ask if there is a finite set $S_{i,\ell}$ of primes of $\mathbf{Z}$ with $\ell \in S_{i,\ell}$ such that the polynomials $$R_{p,i,\ell}(t) = \det(1 - t \phi_p|{\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p},\mathbf{Q}_{\ell}))$$
for all $p \not\in S_{i,\ell}$ are "linked" in the sense that there is a single finite-dimensional continuous $\mathbf{Q}_{\ell}$-linear representation
$$\rho_{i,\ell}: G_{\mathbf{Q},S_{i,\ell}} \rightarrow {\rm{GL}}(V_{i,\ell})$$
of the Galois group over $\mathbf{Q}$ of its maximal extension (inside $\overline{\mathbf{Q}}$) unramified outside $S_{i,\ell}$ such that
$$\det(1 - t \rho_{i,\ell}(\phi_p)|V_{i,\ell}) = R_{p,i,\ell}(t)$$
for all $p \not\in S_{i,\ell}$, where $\phi_p \in G_{\mathbf{Q},S_{i,\ell}}$ is a member of the conjugacy class of geometric Frobenius elements at $p$ (all choices giving the same determinant). This would imply in particular that the degree of $R_{p,i,\ell}(t)$ is the same for all $p \not\in S_{i,\ell}$, but it is a much stronger statement: that $\rho_{i,\ell}$ would be a kind of "$\ell$-adic glue" which unifies the disparate $R_{p,i,\ell}(t)$'s coming from the geometric special fibers $X_{\overline{\mathbf{F}}_p}$ in varying characteristics $p \not\in S_{i,\ell}$.
The crux of the matter then is the following fundamental fact: the continuous representation $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}},\mathbf{Q}_{\ell})$ is such a $\rho_{i,\ell}$, for an appropriate choice of $S_{i,\ell}$. Why? Here is where one has to use a real theorem, namely the preservation of constructibility of $\ell$-adic sheaves under higher direct images with proper support, coupled with the proper base change theorem. More precisely, if $h:Y' \rightarrow Y$ is any separated map of finite type between noetherian schemes over $\mathbf{Z}[1/\ell]$ and if $\mathscr{F}$ is any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y'$ (e.g., the constant sheaf $\mathbf{Q}_{\ell}$) then ${\rm{R}}^i h_{!}(\mathscr{F})$ is a constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ whose formation moreover commutes with any base change (the latter due to the proper base change theorem).
The point is that any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ is lisse over a dense open $U$ (depending on the sheaf), and hence "is" just a continuous $\mathbf{Q}_{\ell}$-linear representation of the fundamental group $\pi_1(U,\eta)$ if $Y$ is normal and connected (with geometric generic point $\eta$). In particular, when $Y$ is a connected Dedekind scheme then over $U$ this lisse sheaf is nothing more or less than an $\ell$-adic representation $\rho$ of the absolute Galois group of the function field of $Y$ (i.e., the residue field at the generic point of $Y$) such that $\rho$ is unramified at all closed points $u$ of $U$. The Galois representation at $u$ arising from the $u$-stalk of the lisse sheaf coincides with the residual Galois representation arising from $\rho$ on the Galois group at the generic point by virtue of its unramifiedness at $u$ (upon choosing a decomposition group at $u$ in the Galois group at the generic point, which amounts to working with a strict henselization at $u$ inside a separable closure of the function field of $Y$ in order to compute the specialization homomorphism from geometric stalk at $u$ to a geometric generic stalk).
For example, take $Y' = X_{\mathbf{Z}[1/\ell]}$ and $Y = {\rm{Spec}}(\mathbf{Z}[1/\ell])$ and $\mathscr{F} = \mathbf{Q}_{\ell}$. The above says that there is a dense open subscheme $U_{i,\ell} \subset {\rm{Spec}}(\mathbf{Z}[1/\ell])$ such that the constructible ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})$ on ${\rm{Spec}}(\mathbf{Z}[1/\ell])$ has restriction over $U_{i,\ell}$ that is lisse. Letting $S_{i,\ell}$ be the finite set of closed points of ${\rm{Spec}}(\mathbf{Z})$ complementary to $U_{i,\ell}$, we have that $\pi_1(U_{i,\ell}) = G_{\mathbf{Q},S_{i,\ell}}$ (using geometric generic point as base point of $\pi_1$) and the lisse restriction ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})|_{U_{i,\ell}}$ has respective stalks at the chosen geometric generic point and geometric closed point at $p \not\in S_{i,\ell}$ identified as Galois modules (for $\mathbf{Q}$ and $\mathbf{F}_p$ respectively) with the respective geometric fibral cohomologies $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}}, \mathbf{Q}_{\ell})$ and ${\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p},\mathbf{Q}_{\ell})$ (recovering in particular that $V_{i,\ell}$ is unramified at $p$, as we know it must be due to $V_{i,\ell}$ arising from a $\pi_1(U_{i,\ell})$-representation).
In other words, it is precisely the lisse pullback of ${\rm{R}}^if_{!}(\mathbf{Q}_{\ell})$ over ${\rm{Spec}}(\mathbf{Z}_{(p)})$ viewed as a representation of $\pi_1({\rm{Spec}}(\mathbf{Z}_{(p)}))$ which is the "$\ell$-adic glue" that links up the $i$th factor in the $\ell$-adic alternating product formula for $\zeta(X_{\overline{\mathbf{F}}_p},t)$ with the single entity $V_{i,\ell}$ that "doesn't know $p$". And the mechanism of this linkage is that (up to conjugation ambiguity!) we can compute that $\pi_1$ using geometric base points over either the generic or closed points of ${\rm{Spec}}(\mathbf{Z}_{(p)})$.
So the upshot is that the lisse restriction of ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})$ over some dense open subscheme of ${\rm{Spec}}(\mathbf{Z}[1/\ell])$ ensures that $V_{i,\ell}$ as built from the cohomology of the geometric generic fiber (no mention of $p$!) is the origin of "uniformity in $p$" when we stare at the $p$-factors of the zeta function of $X$ for varying $p$ (away from some finite set of primes). Note in particular that the set of "bad" primes here is not encoded by geometric means via "good reduction" (a bad notion to consider away from the proper case anyway); it's all about finding a dense open inside ${\rm{Spec}}(\mathbf{Z}[1/\ell])$ over which the constructible ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})$ has lisse restriction.
Note in particular that each $V_{i,\ell}$ on its own does not have anything to do with point-counting (away from special cases like curves and abelian varieties). It is only the alternating product built from these which is related to point-counting. But it is the $V_{i,\ell}$'s which are where the action is.
The above is thoroughly $\ell$-adic for each $\ell$ separately whereas the zeta functions above do not mention $\ell$, so a truly satisfying sense of "uniformity in $p$" (away from a finite exceptional set) would be given by proving two more things: $U_{i,\ell}$ is "independent of $\ell$" in the sense that $U_{i,\ell} = U_i - \{\ell\}$ for some single dense open $U_i \subset
{\rm{Spec}}(\mathbf{Z})$ and that the $V_{i,\ell}$ for varying $\ell$ constitute a "compatible family" in the sense defined in Serre's book Abelian $\ell$-adic representations (here, it would mean that for $p$ corresponding to a closed point of $U_i$ and any $\ell \ne p$ the characteristic polynomial of $\phi_p$ on $V_{i,\ell}$ lies in $\mathbf{Q}[t]$ and is independent of such $\ell$).
If $X_{\mathbf{Q}}$ were smooth and proper over $\mathbf{Q}$, so $X_{\mathbf{Z}[1/N]}$ is smooth and proper over $\mathbf{Z}[1/N]$ for sufficiently divisible $N > 0$, then the smooth and proper base change theorems would ensure that we could take $U = {\rm{Spec}}(\mathbf{Z}[1/N])$ and the Riemann Hypothesis would provide the "compatible family" aspect (essentially because it rules out cancellation in the alternating $\ell$-adic formula, combined with the zeta function being unaware of $\ell$). But beyond that case we don't know: "independence of $\ell$" for the characteristic polynomial of Frobenius acting on the $i$th compactly supported $\ell$-adic cohomology of a separated finite type $\mathbf{F}_p$-scheme is believed to be true but remains an unsolved problem.
(If you look at the Introduction to deJong's IHES paper on alterations you'll see that he was initially hopeful that his results replacing absence of resolutions of singularities in positive characteristic might have applications to prove new "independence of $\ell$" results, but that this didn't pan out; I am not aware of anyone having made substantial progress on it since that time either, but would be happy to hear to the contrary. Even if we grant resolution of singularities then I don't think an implication is known. In the absence of precise control on weights as the purity provided by RH in the smooth proper case, it is hard to geometrically isolate the contribution in a single cohomological degree from the rest, as the long exact excision sequence associated to a stratification lumps together all cohomological degrees. Deligne's Weil II is very suggestive, but alas I think not enough even assuming resolution.)
