Do you really mean to consider an abelian scheme over the *entire* ring of integers, and not just a localization thereof? Either way, every finite flat group scheme $G$ over a domain $R$ with fraction field $F$ such that ${\rm{char}}(F)$ does not divide the order $n$ of $G$ is etale over $R[1/n]$ (as can be checked on geometric fibers, using that the identity component on such fibers has order which divides $n$ yet is a multiple of the residue characteristic, forcing the identity component to be trivial and hence the geometric fiber to be etale at the identity and therefore etale everywhere via translation). Thus, as for any finite etale scheme with constant fiber rank over any base at all, it becomes constant over a finite etale cover of $R[1/n]$ (using base change against itself to split off the diagonal and then induct on fiber-degree).