This indeed a strong condition! But to get a sense of it, let us look at the simplest case
where $X$ is a smooth not necessarily projective complex curve. Also let
$\mathcal{F}$ be locally constant, so we can identify it with a representation of $\pi_1(X)$.  The vanishing condition will imply that the topological
Euler characteristic must be zero. So either $X= G_m$ or $X$ is an elliptic curve.
In both cases, nontrivial examples exist: any local system with
$$H^0(X,\mathcal{F})= \mathcal{F}^{\pi_1(X)}=0$$
will work. In the first case, this is clear because the Euler characteristic of $\mathcal{F}$ is zero and there is only $H^1$ to contend with. In the second, you use Poincare duality to also kill $H^2$.

I think this can be pushed to apply to any local system satisfying the above condition on a semiabelian variety (extension of an abelian variety by a torus)