Let $X$ be a smooth projective variety over $\mathbb{Q}$. Choose a model $\mathcal{X}$ over $\mathbb{Z}$.
Can it be that for infinitely many primes $p$ there are non-trivial automorphisms of $\mathcal{X}\otimes \overline{\mathbb{F}_p}$ and for infinitely many primes there aren't?
Maybe it happens for K3 surfaces but I can't think of an example.
Let $E$ be an elliptic surface defined by the equation $y^2 = x^3 + A(t) x + B(t)$ with $A$ a polynomial of degree $12$ and $B$ a polynomial of degree $18$, generic except for the condition that $1^3 + A(0) 1 + B(0)=1$, and let $X$ be obtained by blowing up $E$ at the point $x=y=1, t=0$.
Then I claim $X$ is an example, at least under a suitable special case of function field BSD. I would not be surprised if there is a simpler example. Furthermore, I expect it is probably possible to remove this assumption somehow.
At all but finitely many primes $p$, $X_{\mathbb F_p}$ can also be described as the blowup of a smooth projective elliptic surface at a point. Let's study its automorphism group.
The map $X \to E \to \mathbb P^1$ can be obtained by viewing $\mathbb P^1$ as the projectivization of the ring of global sections of powers of the canonical bundle, and thus is invariant under automorphisms. So any automorphism must give an automorphism of $\mathbb P^1$ preserving the $j$ invariant of the fiber. For $A,B$ sufficiently generic, no nontrivial such automorphisms of $\mathbb P^1$ will exist, so we can assume the automorphism fixes $\mathbb P^1$.
The automorphism must act on the Néron model $E$ as a surface over $\mathbb P^1$ and must therefore act (faithfully) on its generic fiber. The automorphisms of the generic fiber $E_{\overline{\mathbb F_p}(t)}$ are, because its $j$ invariant is neither $0$ nor $1728$, the semidirect product of $\mathbb Z/2$ with the group of sections $E( \overline{\mathbb F_p}(t))$.
If $E( \overline{\mathbb F_p}(t))$ is trivial, then the only automorphisms of $E$ is the negation $y \to \pm y$, which doesn't fix the blown-up point and therefore doesn't lift to an automorphism of $X$, so the automorphism group is trivial.
If $E( \overline{\mathbb F_p}(t))$ has positive rank, then $E( \overline{\mathbb F_p}(t)$ defines by translation a group of automorphisms of the Neron model. Passing to some finite field $\mathbb F_q$ over which one of these automorphisms is defined, we see that some power of it fixes the point $x=y=1,t=0$, as that point has a finite orbit $E_0 (\mathbb F_q)$, and that power therefore lifts to a nontrivial automorphism of the blow-up $X$.
For $E$ sufficiently generic, we won't have any torsion sections, so one of these two cases must happen. It suffices to show each occurs infinitely often.
For $\pi \colon E \to \mathbb P^1$, $H^1(\mathbb P^1_{\overline{\mathbb Q}} , R^1 \pi_* \mathbb Q_\ell(1))$ is a Galois representation of dimension $32$ such that the characteristic polynomial of Frobenius at each prime $p$ matches the $L$-function of the elliptic curve $E$ over $\mathbb F_p(t)$. For $E$ sufficiently generic, the monodromy of this Galois representation will be the full orthogonal group. So half the primes have Frobenius with determinant $1$ and half will have Frobenius with determinant $-1$.
Using Frobenius torus theory, one can check that almost all the primes of determinant $1$ will have the characteristic polynomial of Frobenius irreducible and almost all the primes of determinant $-1$ will have the characteristic polynomial of Frobenius irreducible except for a single root at $1$ and a single root at $-1$. Furthermore, we can assume that these irreducible polynomials are not equal to an irreducible cyclotomic polynomial.
In the first case, it follows for all $n$ that the $L$-function of $E$ over $\mathbb F_{p^n}(t)$ does not vanish at $1$, and therefore, by the easy case of function field BSD, $E$ has rank $0$ over $\mathbb F_{p^n}(t)$. Thus $E$ has rank $0$ over $\overline{\mathbb F_p}(t)$ and we are done.
In the second case, it follows that the $L$-function of $E$ over $\mathbb F_p(t)$ has order of vanishing $1$ at $1$, and therefore, assuming function field BSD, $E$ has rank $1$ over $\mathbb F_p(t)$, thus positive rank over $\overline{\mathbb F_p}(t)$, and we are done.
$\newcommand{\bQ}{\mathbb{Q}}\newcommand{\bZ}{\mathbb{Z}}\newcommand{\fp}{\mathfrak{p}}\newcommand{\bF}{\mathbb{F}}\newcommand{\bP}{\mathbb{P}}$Here is a variation on the theme of Will Sawin's answer which allows for a non-conditional example. The construction is based on the following theorem of François Charles:
Theorem (Theorem 1.1 in Exceptional isogenies between reductions of pairs of elliptic curves,) If $E_1,E_2$ are elliptic curves over a number field $K$ then there exist infinitely many primes $\mathfrak{p}$ of $K$ such that the reductions $E_{1,\mathfrak{p}}$ and $E_{2,\mathfrak{p}}$ are isogenous over $\overline{\mathbb{F}}_p$.
So let $E_1,E_2$ be elliptic curves over $\bQ$ that are not geometrically isogenous. There must be infinitely many primes at which the reductions of $E_1, E_2$ are not geometrically isogenous, as otherwise Chebotarev density would imply that there is a finite extension $K/\mathbb{Q}$ over which Galois representations $V_lE_1,V_lE_2$ are isomorphic, contradicting the assumption.
Charles's theorem says, on the other hand, that there are also infinitely many primes $p$, such that $E_{1,p}$ and $E_{2,p}$ are geometrically isogenous. For the variety $Y=E_1\times E_2$ the reduction $Y_p$ has more automorphism when $E_{1,p}$ and $E_{2,p}$ are isogenous than for other primes $p$. We are going to rigidify $Y$ in a way that there will be no non-trivial automorphisms modulo those primes for which $E_{1,p}$ and $E_{2,p}$ are non-isogenous while keepong an infinite automorphism group for the rest of the primes of good reduction.
We'll start by replacing $\bQ$ by a finite extension and will construct the desired variety there, so that the ultimate example will be obtained by considering that variety as a (non-geometrically connected) scheme over $\bQ$. Let $K/\bQ$ be a finite extension such that $E_1[3](K)=(\bZ/3)^2,E_2[3](K)=(\bZ/3)^2$.
Pick an injective function $a:(E_1\times E_2)[3](K)\to \bZ_{\geq 1}$ and define the variety $X$ as the following birational modification of the surface $Y_K$: for each $3$-torsion point $x\in (E_1\times E_2)[3](K)$ perform a blow-up of $E_1\times E_2$ at an infinitely near point of order $a(x)$ supported at $x$. The fiber of the map $X\to Y_K$ at a point $x$ is thus a chain of $a(x)$ copies of the projective line. There is also a natural integral model $\mathcal{X}$ of $X$ over $\mathcal{O}_K$, constructed by blowing up the product of models of $E_1$ and $E_2$.
Lemma If $\fp\not\mid 3$ is a prime of $K$ such that $E_1,E_2$ have good reduction at $\fp$, then $X_{\fp}\times\overline{\bF}_p$ has non-trivial automorphisms if and only if $E_{1,\fp}$ and $E_{2,\fp}$ are geometrically isogenous.
Proof. An automorphism $g$ of $X_{\overline{\bF}_p}$ induces a permutation of closed subvarieties of this surface that are isomorphic to $\bP^1$, preserving the incidence relation between them. In particular, for each $x\in Y[3](\overline{\bF}_p)$ the fiber of $X\to Y$ above $x$ is preserved by $g$. An automorphism of a chain of $\bP^1$s must have a fixed point, so $g$ admits a fixed point lying in the fiber of $X\to Y$ above $0\in Y_{\overline{\bF}_p}=E_{1,\overline{\bF}_p}\times E_{2,\overline{\bF}_p}$ and therefore $g$ descends to an automorphism of $Y_{\overline{\bF}_p}$ because $Y_{\overline{\bF}_p}$ can be established as the Albanese variety of $X_{\overline{\bF}_p}$ wrt that fixed point.
If $E_{1,\overline{\bF}_p}$ and $E_{2,\overline{\bF}_p}$ are not isogenous then any automorphism $h$ of $Y_{\overline{\bF}_p}$ must be a product of automorphisms of $E_{1,\overline{\bF}_p}$ and $E_{2,\overline{\bF}_p}$ and, in particular, has finite order. For it to lift to an automorphism of $X_{\fp}$ the action of $h$ on $Y_{\fp}[3]$ must be trivial, but this forces $h$ to be trivial because any matrix in $GL_n(\bZ_3)$ congruent to $1$ mod $3$ has infinite order if it is not the identity.
Conversely, suppose that $f:E_{1,\overline{\bF}_p}\to E_{2,\overline{\bF}_p}$ is an isogeny. Let $N$ be an integer larger than any of the values of the function $a$. Then $g:(t,s)\mapsto (t,s+3p^N\cdot f(t))$ is an automorphism of $Y_{\overline{\bF}_p}$ that preserves every $3$-torsion point and, moreover, for every $3$-torsion point $x\in Y_{\fp}[3]$ the action of $g$ on the quotient $\mathcal{O}_{Y,x}/\mathfrak{m}_x^N$ is trivial. Therefore $g$ lifts to an automorphism of the blow-up $X_{\overline{\bF}_p}\to Y_{\overline{\bF}_p}$, as desired.
