Extension of **Daniel Loughan**'s example (which is also known, but not as
well-known as it should(?) be):  if a prime $p$ is of the form $dn+1$ then
the Fermat hypersurface $\sum_{i=1}^d x_i^d$ in ${\bf P}^{d-1}({\bf F}_q)$
is smooth and its number of rational points is not congruent to $1 \bmod p$. 
Indeed the usual argument for Chevalley(-Warning) shows that
the number of rational points is congruent mod $p$ to $1 \pm t$
where $t$ is the $(x_1 x_2 \cdots x_d)^{p-1}$ coefficient of
$\left(\sum_{i=1}^d x_i^d\right)^{p-1}$, and when $p = dn+1$
this coefficient is $(p-1)!/n!^d$ which is clearly not $0 \bmod p$.