Apologies; my first reading of the question (which confused an $n$ for a $p$) assumed you were only given $\#X(\mathbf{F}_p)$ for $p$ in a certain range. If you are given (almost) all the $\#X(\mathbf{F}_p)$ (the case $N = 1$) then you can determine the Betti numbers, as mentioned in the first version of this answer, given with slightly more detail below.

The point counts determine, by the Lefschetz trace formula, the trace of Frobenius at $p$ (for good primes $p$) acting on the virtual Galois representation
$$[V_l]:=\sum (-1)^i [H^i(X,\mathbf{Q}_l)]$$
(in the Grothendieck group of Galois representations over $\mathbf{Q}_l$) for any fixed $l$. By the Chebotarev density theorem, as long as you include almost all primes $p$ these Frobenii are dense, and hence this determines the trace any *any* element in the Galois group on this virtual representation $[V_l]$. By the Brauer-Nesbitt Theorem, the set of traces determines the virtual representation (in the Grothendieck group). By purity, you can then determine (the semisimplifications of) the Galois representations $H^i(X,\mathbf{Q}_l)$ completely, and hence also the Betti numbers.