I actually have a paper on a very closely related topic, at least for principally polarised abelian varieties:

Daniel Loughran, Gregory Sankaran- Rationality and arithmetic of the moduli of abelian varieties, https://arxiv.org/abs/2310.01244

Firstly for cardinality reasons, it is more natural to ask rather that you can approximate an abelian variety over $\mathbb{Q}_p$ by one over $\mathbb{Q}$ (by approximate, I mean that the reductions modulo $p^n$ are isomorphic for some large $n$).

The answer is: You can approximate with something over $\mathbb{Q}$ in dimensions 1,2,3. It is unknown if it is possible in dimensions $4,5,6$. It is not possible in dimensions at least $7$ assuming the Bombieri-Lang conjecture.

Without any kind of polarisation I really have no idea. It is difficult to attack such problems without a reasonable moduli space parametrising the objects of interest.

As phrased, this problem looks just as difficult as determining whether a given element of $\mathbb{Q}_p$ lies in $\mathbb{Q}$. There are real numbers which are unknown to be rational (e.g. $\pi + e$); I don't see why the $p$-adic case should be substantially easier.

In light of this, it is more natural to ask rather that you can approximate an abelian variety over $\mathbb{Q}_p$ by one over $\mathbb{Q}$ (by approximate, I mean that the reductions modulo $p^n$ are isomorphic for some large $n$).

This problem is considered in the following paper, at least for principally polarised abelian varieties:

Daniel Loughran, Gregory Sankaran- Rationality and arithmetic of the moduli of abelian varieties, https://arxiv.org/abs/2310.01244

The answer is: You can approximate with something over $\mathbb{Q}$ in dimensions 1,2,3. It is unknown if it is possible in dimensions $4,5,6$. It is not possible in dimensions at least $7$ assuming the Bombieri-Lang conjecture.

Without any kind of polarisation I really have no idea. It is difficult to attack such problems without a reasonable moduli space parametrising the objects of interest.