Consider a simple abelian variety $A/\mathbb{C}$ with sufficiently many CMs by $\mathcal{O}$, where $\mathcal{O}$ is an order in a CM field $K$. Specifically, $K$ is a CM field of degree $2g$, where $g = \dim A$. Let $(K,\, \Phi)$ be the CM type. It can be shown that there is a field $F$ such that $A$ is defined and has smCM over $F$.
Is there an explicit field $F_0(\mathcal{O},\, \Phi)$ such that $A$ is defined over $F_0$, with CM over $F_0$?
As motivation: If $A$ is an elliptic curve, $K$ an imaginary quadratic field, and $\mathcal{O}$ is the ring of integers in $K$, then the answer is yes: $A$ is defined with CM over the Hilbert class field $H(K)$, and moreover there is no better field of definition, so $F_0$ can be taken to be $H(K)$. It is moreover my understanding that if $\mathcal{O}$ is a different order, then $F_0$ can be taken to be the Hilbert class field of the order $\mathcal{O}$, though this is something I heard and might be misinformed.
My question, then, is whether this extends to higher-dimensional abelian varieties. It's known that such an abelian variety can be defined over some number field, but the arguments I've seen in this direction are of the form "if $A$ is defined over $\mathbb{C}$, then it can be defined over a finite-type ring over $\mathbb{Q}$, so reduce modulo a maximal ideal and check that it works." There is, on the other hand, an old result of Taniyama-Shimura that says: if $A$ has CM type $(K,\, \Phi)$ and $A$ is defined over $F$, then $F$ contains the reflex field $E(K,\, \Phi)$.
So, can the result: An elliptic curve $A/\mathbb{C}$ with CM by $\mathcal{O}$ can be defined over the Hilbert class field of $\mathcal{O}$ be extended, in some form, to higher-dimensional abelian varieties? And if so, in what form?
Thanks in advance.