I actually have a paper on a very closely related topic, at least for principally polarised abelian varieties:

Daniel Loughran, Gregory Sankaran- Rationality and arithmetic of the moduli of abelian varieties, https://arxiv.org/abs/2310.01244

Firstly for cardinality reasons, it is more natural to ask rather that you can approximate an abelian variety over $\mathbb{Q}_p$ by one over $\mathbb{Q}$ (by approximate, I mean that the reductions modulo $p^n$ are isomorphic for some large $n$).


The answer is: You can approximate with something over $\mathbb{Q}$ in dimensions 1,2,3. It is unknown if it is possible in dimensions $4,5,6$. It is not possible in dimensions at least $7$ assuming the Bombieri-Lang conjecture.

Without any kind of polarisation I really have no idea. It is difficult to attack such problems without a reasonable moduli space parametrising the objects of interest.