The $j$ invariant gives an isomorphism over $\mathbb Z$,
$$j:\mathcal A_1\to\mathbb A^1,$$
of the moduli space of elliptic curves. So $j(E)\in\overline{\mathbb Z}$ can be interpreted as saying that $\langle E\rangle\in \mathcal A_1(\overline{\mathbb Z})$, where I've written $\langle E\rangle$ for the isomorphism class of $E$. As noted by Dror, there are Igusa invariants which (largely) do the same thing for $\mathcal A_2$. But in general, probably the right interpretation is that one should look at the moduli space $\mathcal A_g/\mathbb Z$, and then it is a theorem that an abelian variety $A$ that has complex multiplication satisfies
$$ \langle A\rangle \in \mathcal A_g(\overline{\mathbb Z}). $$
Another useful interpretation is that the elements of $\mathcal A_g(\overline{\mathbb Z})$ have potential good reduction, so in particular this is true for CM abelian varieties. There is also the criterion of Neron-Ogg-Shafarevich that you might find relevant; see
Serre, Jean-Pierre; Tate, J., Good reduction of abelian varieties, Ann. Math. (2) 88, 492-517 (1968). ZBL0172.46101.