This is an elaboration on ACL's answer, way too long for a comment, which highlights a technical ingredient (well-known to all experts) that underlies the precise sense in which the $\ell$-adic etale cohomology of the geometric generic fiber provides a "uniformity" in $p$: the good properties of constructible $\ell$-adic sheaves.
In particular, I think it is a mistake to try to understand a precise sense of "uniformity in $p$" by focusing on point-counting: this misses the key structure, as noted in ACL's answer, namely certain $\ell$-adic representations (of the absolute Galois group of $\mathbf{Q}$) which individually are not expressed via point-counting at all. 

To explain this requires some preparations, hence the length of what follows (which is all standard stuff, but perhaps hard to extract for a non-expert; maybe even what follows is hard to read in parts for a non-expert, but I think it is important to recognize where serious theorems of etale cohomology are doing some work, going beyond the cohomological formula for the zeta function of a single variety over a single finite field).  The crux is that the robustness of constructibility provides the magical glue linking behavior at different primes.

----

Literally from the product definition, the zeta function of a separated finite type $\mathbf{Z}$-scheme $X$ is the product $\prod_p \zeta(X_{\mathbf{F}_p}, p^{-s})$ of the zeta functions of the fibers, with ${\rm{Re}}(s)$ is sufficiently large (determined by fiber dimensions alone; see Serre's article in the Purdue conference proceedings on arithmetic geometry from the mid-1960's). By the work of Dwork or Grothendieck-Deligne, the zeta function of any separated scheme of finite type over $\mathbf{F}_p$ (such as $X_{\mathbf{F}_p}$) is a rational function in $p^{-s}$. 

The cohomological formalism provides an "$\ell$-adic" explanation for the rationality of the factor at each prime $p$ in the sense that
for any prime $\ell \ne p$ we have 
$$\zeta(X_{\mathbf{F}_p}, t) = \prod_{i\ge 0} \det(1 - \phi_p t|
{\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p}, \mathbf{Q}_{\ell}))^{(-1)^{i+1}}$$
in $\mathbf{Q}_{\ell}[\![t]\!]$, where the left side is initially just a formal power series in $1 + t \mathbf{Z}[\![t]\!]$ (defined as a product over closed points of $X_{\mathbf{F}_p}$) and the rational function over $\mathbf{Q}_{\ell}$ on the right side might involve internal cancellations among the various determinant polynomials (ruled out for smooth proper $X_{\mathbf{F}_p}$ by the Deligne's work on the Riemann Hypothesis, but not otherwise).  In other words, the "$\ell$-adic" explanation for rationality rests on the fact that $\mathbf{Q}(\!(t)\!) \cap \mathbf{Q}_{\ell}(t) = \mathbf{Q}(t)$ inside $\mathbf{Q}_{\ell}(\!(t)\!)$ (and Dwork's approach provides a variant of that explanation with $\ell=p$).  In the displayed product on the right side, $i$ goes up to $2 \dim X_{\mathbf{F}_p}$ (which is bounded independently of $p$, and in fact equal to $2 \dim X_{\mathbf{Q}}$ for all but finitely many $p$).

That was all just setup.  Now *fix* a prime $\ell$ and an integer $i \ge 0$. One can ask if there is a finite set $S_{i,\ell}$ of primes of $\mathbf{Z}$ with $\ell \in S_{i,\ell}$ such that the polynomials $$R_{p,i,\ell}(t) = \det(1 - \phi_p t|{\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p},\mathbf{Q}_{\ell}))$$
for all $p \not\in S_{i,\ell}$ are "linked" in the sense that there is a single finite-dimensional continuous $\mathbf{Q}_{\ell}$-linear representation 
$$\rho_{i,\ell}: G_{\mathbf{Q},S_{i,\ell}} \rightarrow {\rm{GL}}(V_{i,\ell})$$
of the Galois group over $\mathbf{Q}$ of its maximal extension (inside $\overline{\mathbf{Q}}$) unramified outside $S_{i,\ell}$  such that
$$\det(1 - t \rho_{i,\ell}(\phi_p)|V_{i,\ell}) = R_{p,i,\ell}(t)$$
for *all* $p \not\in S_{i,\ell}$, where $\phi_p \in G_{\mathbf{Q},S_{i,\ell}}$ is a member of the conjugacy class of geometric Frobenius elements at $p$ (all choices giving the same determinant).  This would imply in particular that the degree of $R_{p,i,\ell}(t)$ is the same for all $p \not\in S_{i,\ell}$, but it is a *much stronger* statement: that $\rho_{i,\ell}$ would be a kind of "$\ell$-adic glue" which unifies the disparate $R_{p,i,\ell}(t)$'s coming from the geometric special fibers $X_{\overline{\mathbf{F}}_p}$ in varying characteristics $p \not\in S_{i,\ell}$.

The crux of the matter then is the following fundamental fact: the continuous representation $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}},\mathbf{Q}_{\ell})$ is such a $\rho_{i,\ell}$, for an appropriate choice of $S_{i,\ell}$.  Why?  Here is where one has to use a real theorem, namely the preservation of constructibility of $\ell$-adic sheaves under higher direct images with proper support, coupled with the proper base change theorem.  More precisely, if $h:Y' \rightarrow Y$ is any separated map of finite type between noetherian schemes over $\mathbf{Z}[1/\ell]$ and if $\mathscr{F}$ is any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y'$ (e.g., the constant sheaf $\mathbf{Q}_{\ell}$) then ${\rm{R}}^i h_{!}(\mathscr{F})$ is a *constructible* $\mathbf{Q}_{\ell}$-sheaf on $Y$ whose formation moreover *commutes with any base change* (the latter due to the proper base change theorem).  The point is that any constructible $\mathbf{Q}_{\ell}$-sheaf on $Y$ is *lisse* over a dense open $U$ (depending on the sheaf), and hence "is" just a continuous $\mathbf{Q}_{\ell}$-linear representation of the fundamental group $\pi_1(U,\eta)$ if $Y$ is *normal* and connected (with geometric generic point $\eta$).  In particular, when $Y$ is a connected Dedekind scheme then over $U$ this lisse sheaf is nothing more or less than an $\ell$-adic representation $\rho$ of the absolute Galois group of the function field of $Y$ (i.e., the residue field at the generic point of $Y$) such that $\rho$ is *unramified* at all closed points $u$ of $U$. The Galois representation at $u$ arising from the $u$-stalk of the lisse sheaf coincides with the residual Galois representation arising from $\rho$ on the Galois group at the generic point by virtue of its unramifiedness at $u$ (upon choosing a decomposition group at $u$ in the Galois group at the generic point, which amounts to working with a strict henselization at $u$ inside a separable closure of the function field of $Y$ in order to compute the specialization homomorphism from geometric stalk at $u$ to a geometric generic stalk).

----

For example, take $Y' = X_{\mathbf{Z}[1/\ell]}$ and $Y = {\rm{Spec}}(\mathbf{Z}[1/\ell])$ and $\mathscr{F} = \mathbf{Q}_{\ell}$.  The above says that there is a dense open subscheme $U_{i,\ell} \subset {\rm{Spec}}(\mathbf{Z}[1/\ell])$ such that the constructible ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})$ on ${\rm{Spec}}(\mathbf{Z}[1/\ell])$ has restriction over $U_{i,\ell}$ that is *lisse*. Letting $S_{i,\ell}$ be the finite set of closed points of ${\rm{Spec}}(\mathbf{Z})$ complementary to $U_{i,\ell}$, we have that $\pi_1(U_{i,\ell}) = G_{\mathbf{Q},S_{i,\ell}}$ (using geometric generic point as base point of $\pi_1$) and the  *lisse* restriction ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})|_{U_{i,\ell}}$ has respective stalks at the chosen geometric generic point and geometric closed point at $p \not\in S_{i,\ell}$ identified *as Galois modules* (for $\mathbf{Q}$ and $\mathbf{F}_p$ respectively) with the respective geometric fibral cohomologies $V_{i,\ell} := {\rm{H}}^i_c(X_{\overline{\mathbf{Q}}}, \mathbf{Q}_{\ell})$ and ${\rm{H}}^i_c(X_{\overline{\mathbf{F}}_p},\mathbf{Q}_{\ell})$ (recovering in particular that $V_{i,\ell}$ is unramified at $p$, as we know it must be due to $V_{i,\ell}$ arising from a $\pi_1(U_{i,\ell})$-representation).  In other words, it is precisely the lisse pullback of ${\rm{R}}^if_{!}(\mathbf{Q}_{\ell})$ over ${\rm{Spec}}(\mathbf{Z}_{(p)})$ viewed as a representation of $\pi_1({\rm{Spec}}(\mathbf{Z}_{(p)}))$ which is the "$\ell$-adic glue" that links up the $i$th factor in the $\ell$-adic alternating product formula for $\zeta(X_{\overline{\mathbf{F}}_p},t)$ with the single entity $V_{i,\ell}$ that "doesn't know $p$".  And the mechanism of this linkage is that (up to conjugation ambiguity!) we can compute that $\pi_1$ using geometric base points over *either* the generic or closed points of ${\rm{Spec}}(\mathbf{Z}_{(p)})$.

So the upshot is that the lisse restriction of ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})$ over some dense open subscheme of ${\rm{Spec}}(\mathbf{Z}[1/\ell])$ ensures that $V_{i,\ell}$ as built from the cohomology of the geometric generic fiber (no mention of $p$!) is the origin of "uniformity in $p$" when we stare at the $p$-factors of the zeta function of $X$ for varying $p$ (away from some finite set of primes). Note in particular that the set of "bad" primes here is *not* encoded by geometric means via "good reduction" (a bad notion to consider away from the proper case anyway); it's all about finding a dense open inside ${\rm{Spec}}(\mathbf{Z}[1/\ell])$ over which the constructible ${\rm{R}}^i f_{!}(\mathbf{Q}_{\ell})$ has lisse restriction.

Note in particular that each $V_{i,\ell}$ on its own **does not have anything to do with point-counting**. It is only the alternating product built from these which is related to point-counting. But it is the $V_{i,\ell}$'s which are where the action is.

-----

The above is thoroughly $\ell$-adic for each $\ell$ separately whereas the zeta functions above do not mention $\ell$, so a truly satisfying sense of "uniformity in $p$" (away from a finite exceptional set) would be given by proving two more things: $U_{i,\ell}$ is "independent of $\ell$" in the sense that $U_{i,\ell} = U_i - \{\ell\}$ for some single dense open $U_i \subset 
{\rm{Spec}}(\mathbf{Z})$ and that the $V_{i,\ell}$ for varying $\ell$ constitute a "compatible family" in the sense defined in Serre's book *Abelian $\ell$-adic representations* (here, it would mean that for $p$ corresponding to a closed point of $U_i$ and any $\ell \ne p$ the characteristic polynomial of $\phi_p$ on $V_{i,\ell}$ lies in $\mathbf{Q}[t]$ and is independent of such $\ell$).

If $X_{\mathbf{Q}}$ were smooth and proper over $\mathbf{Q}$, so $X_{\mathbf{Z}[1/N]}$ is smooth and proper over $\mathbf{Z}[1/N]$ for sufficiently divisible $N > 0$, then the smooth and proper base change theorems would ensure that we could take $U = {\rm{Spec}}(\mathbf{Z}[1/N])$ and the Riemann Hypothesis would provide the "compatible family" aspect (essentially because it rules out cancellation in the alternating $\ell$-adic formula, combined with the zeta function being unaware of $\ell$).  But beyond that case we don't know: "independence of $\ell$" for the characteristic polynomial of Frobenius acting on the $i$th compactly supported $\ell$-adic cohomology of a separated finite type $\mathbf{F}_p$-scheme is believed to be true but remains an unsolved problem.