In the paper of Mazur, Modular curves and the Eisenstein ideal (1977), he showed the following (at page 57):

Lemma (3.4) (Criterion for constancy) Let $G$ be an etale admissible $p$-group over $\text{Spec} (\mathbb{Z}[1/N])$. If $N \not\equiv 1 \mod p$, then $G$ is constant. In general, $G$ is constant if and only if there is a prime number $\ell \neq N$ such that:

a) $\ell$ is not a $p$-th power modulo $N$;

b) The action of $\varphi_\ell$ in the Galois representation of $G$ is trivial.

Here, $p$ and $N$ are distinct prime numbers, and $\varphi_\ell$ denotes the Frobenius morphism.

What happens if $N$ is no longer a prime?

For instance, let $N$ be a positive integer not divisible by an odd prime number $p$. Let $G$ be an etale admissible $p$-group over $\text{Spec} (\mathbb{Z}[1/N])$. Suppose that $\varphi_\ell$ acts trivially on $G$ for infinitely many prime numbers $\ell$. Is then $G$ constant?