Suppose $X$ is proper flat scheme over Sepc$\mathbb{Z}$. If $p$ is a large prime, then by Weil conjectures one can recover the Betti numbers of $X$ from the size of $X(\mathbb{F}_{p^n})$ for all $n$. My question is, what if instead of fix a prime and vary $n$, we fix an $n$ and vary $p$?
More precisely, let $N$ be a fixed number, suppose we know the size of $X(\mathbb{F}_{p^n})$ for all $n$ between 1 and $N$, and (almost) all prime $p$. Can we recover the Betti numbers of $X$?
For the first claim, I think you want to also assume $X$ is smooth (away from some finite set of primes, not including $p$).
For the second claim, again with the assumption that $X$ is smooth, this would follow from a weak form of the generalized Sato-Tate conjecture. But nothing like this is known currently without very strong additional assumptions on $X$.
See Nick Katz's article Simple Things We Don't Know which is about almost exactly this question.
Apologies; my first reading of the question (which confused an $n$ for a $p$) assumed you were only given $\#X(\mathbf{F}_p)$ for $p$ in a certain range. If you are given (almost) all the $\#X(\mathbf{F}_p)$ (the case $N = 1$) then you can determine the Betti numbers, as mentioned in the first version of this answer, given with slightly more detail below.
The point counts determine, by the Lefschetz trace formula, the trace of Frobenius at $p$ (for good primes $p$) acting on the virtual Galois representation $$[V_l]:=\sum (-1)^i [H^i(X,\mathbf{Q}_l)]$$ (in the Grothendieck group of Galois representations over $\mathbf{Q}_l$) for any fixed $l$. By the Chebotarev density theorem, as long as you include almost all primes $p$ these Frobenii are dense, and hence this determines the trace any any element in the Galois group on this virtual representation $[V_l]$. By the Brauer-Nesbitt Theorem, the set of traces determines the virtual representation (in the Grothendieck group). By purity, you can then determine (the semisimplifications of) the Galois representations $H^i(X,\mathbf{Q}_l)$ completely, and hence also the Betti numbers.
