$\newcommand{\bQ}{\mathbb{Q}}\newcommand{\bZ}{\mathbb{Z}}\newcommand{\fp}{\mathfrak{p}}\newcommand{\bF}{\mathbb{F}}\newcommand{\bP}{\mathbb{P}}$Here is a variation on the theme of Will Sawin's answer which allows for a non-conditional example. The construction is based on the following theorem of François Charles:

Theorem (Theorem 1.1 in Exceptional isogenies between reductions of pairs of elliptic curves,) If $E_1,E_2$ are elliptic curves over a number field $K$ then there exist infinitely many primes $\mathfrak{p}$ of $K$ such that the reductions $E_{1,\mathfrak{p}}$ and $E_{2,\mathfrak{p}}$ are isogenous over $\overline{\mathbb{F}}_p$.

So let $E_1,E_2$ be elliptic curves over $\bQ$ that are not geometrically isogenous. There must be infinitely many primes at which the reductions of $E_1, E_2$ are not geometrically isogenous, as otherwise Chebotarev density would imply that there is a finite extension $K/\mathbb{Q}$ over which Galois representations $V_lE_1,V_lE_2$ are isomorphic, contradicting the assumption.

Charles's theorem says, on the other hand, that there are also infinitely many primes $p$, such that $E_{1,p}$ and $E_{2,p}$ are geometrically isogenous. For the variety $Y=E_1\times E_2$ the reduction $Y_p$ has more automorphism when $E_{1,p}$ and $E_{2,p}$ are isogenous than for other primes $p$. We are going to rigidify $Y$ in a way that there will be no non-trivial automorphisms modulo those primes for which $E_{1,p}$ and $E_{2,p}$ are non-isogenous while keepong an infinite automorphism group for the rest of the primes of good reduction.

We replace $\bQ$ by a finite extension and construct the desired variety there, so that the ultimate example will be obtained by considering that variety as a (non-geometrically connected) scheme over $\bQ$. Let $K/\bQ$ be a finite extension such that $E_1[3](K)=(\bZ/3)^2,E_2[3](K)=(\bZ/3)^2$.

Pick an injective function $a:(E_1\times E_2)[3](K)\to \bZ_{\geq 1}$ and define the variety $X$ as the following birational modification of the surface $Y_K$: for each $3$-torsion point $x\in (E_1\times E_2)[3](K)$ perform a blow-up of $E_1\times E_2$ at an infinitely near point of order $a(x)$ supported at $x$. The fiber of the map $X\to Y_K$ at a point $x$ is thus a chain of $a(x)$ copies of the projective line. There is also a natural integral model $\mathcal{X}$ of $X$ over $\bZ$, constructed by blowing up the product of models of $E_1$ and $E_2$.

Lemma If $\fp\not\mid 3$ is a prime of $K$ such that $E_1,E_2$ have good reduction at $\fp$, then $X_{\fp}\times\overline{\bF}_p$ has non-trivial automorphisms if and only if $E_{1,\fp}$ and $E_{2,\fp}$ are geometrically isogenous.

Proof. An automorphism $g$ of $X_{\fp}$ induces a permutation of closed subvarieties of this surface that are isomorphic to $\bP^1$, preserving the incidence relation between them. In particular, for each $x\in Y[3](\overline{\bF}_p)$ the fiber of $X\to Y$ above $x$ is preserved by $g$. An automorphism of a chain of $\bP^1$s must have a fixed point, so $g$ admits a fixed point lying in the fiber of $X\to Y$ above $0\in Y_{\fp}=E_{1,\fp}\times E_{2,\fp}$ and therefore $g$ descends to an automorphism of $Y_{\fp}$ because $Y_{\fp}$ can be established as the Albanese variety of $X_{\fp}$ wrt that fixed point.

If $E_{1,\fp}$ and $E_{2,\fp}$ are not isogenous then any automorphism $h$ of $Y_{\fp}$ must be a product of automorphisms of $E_{1,\fp}$ and $E_{2,\fp}$ and, in particular, has finite order. For it to lift to an automorphism of $X_{\fp}$ the action of $h$ on $Y_{\fp}[3]$ must be trivial, but this forces $h$ to be trivial because any matrix in $GL_n(\bZ_3)$ congruent to $1$ mod $3$ has infinite order if it is not the identity.

Conversely, suppose that $f:E_{1,\overline{\bF}_p}\to E_{2,\overline{\bF}_p}$ is an isogeny. Let $N$ be an integer larger than any of the values of the function $a$. Then $g:(t,s)\mapsto (t,s+3p^N\cdot f(t))$ is an automorphism of $Y_{\overline{\bF}_p}$ that preserves every $3$-torsion point and, moreover, for every $3$-torsion point $x\in Y_{\fp}[3]$ the action of $g$ on the quotient $\mathcal{O}_{Y,x}/\mathfrak{m}_x^N$ is trivial. Therefore $g$ lifts to an automorphism of the blow-up $X_{\overline{\bF}_p}\to Y_{\overline{\bF}_p}$, as desired.