Not a definitive answer, but the question is stale, and this could be at least somewhat useful.
Note that we can rewrite the coefficient in the composition as
$$
\left[\frac{z^n}{a_n}\right] F(G(z)) = \sum\limits_{k=0}^\infty \frac{1}{a_k} \left[\frac{z^k}{a_k}\right] F(z)\left[\frac{z^n}{a_n}\right] G(z)^k,
$$
and for majority of functions of interest we would rather want that groups of summands corresponding to the same partition make up an integer, rather than focusing on an integer total sum. In a very general case, this boils down to a requirement that for any $n$ and a partition $i_1+\dots+i_k = n$, it holds that, assuming $j_t$ is the number of blocks of size $t$, the number
$$
A(i_1,\dots,i_k) = \frac{1}{a_k} \binom{k}{j_1,\dots,j_k} \frac{a_n}{a_{i_1} \dots a_{i_k}}
$$
is an integer. I suppose, here are some useful properties about such sequences:
- For $\left[\frac{z^n}{a_n}\right]G(z)=g_n$ and $\left[\frac{z^n}{a_n}\right]F(z)=f_n$, the composition rewrites as
$$
\left[\frac{z^n}{a_n}\right] F(G(z)) = \sum\limits_{k=0}^\infty f_k \sum\limits_{\substack{i_1+\dots+i_k=n\\i_1\leq\dots\leq i_k}} A(i_1,\dots,i_k) g_{i_1}\dots g_{i_k},
$$
meaning that $A(i_1,\dots,i_k)$ combinatorially stands for the weight with which the corresponding partition of $n$ into $k$ blocks of sizes $i_1,\dots,i_k$ is counted. Typically, this number represents the number of ways to distribute $n$ atoms among the $k$ blocks and/or build some structure on top of the blocks.
- For $a_n = n!$, we have $A(i_1,\dots,i_k) = \frac{1}{j_1!\dots j_k!} \binom{n}{i_1,\dots,i_k}$, which has combinatorial meaning as the number of partitions of $n$ into blocks of size $i_1,\dots,i_k$, where order of blocks doesn't matter.
- For $a_n = x^{n-1}$, we have $A(i_1,\dots,i_k) = \binom{k}{j_1,\dots,j_k}\frac{1}{x^{k-1}} \frac{x^{n-1}}{x^{n-k}}=\binom{k}{j_1,\dots,j_k}$.
- For $a_n = x^{\binom{n}{2}}$, we have
$$
A(i_1,\dots,i_k) = \binom{k}{j_1,\dots,j_k}x^{\binom{n}{2} -\binom{i_1}{2}-\dots-\binom{i_k}{2}-\binom{k}{2}}.
$$
At the same time, $\binom{a+b}{2} - \binom{a}{2} - \binom{b}{2} = ab$, from which we can rewrite
$$
A(i_1,\dots,i_k) = \binom{k}{j_1,\dots,j_k}\prod\limits_{a < b} x^{i_a i_b-1},
$$
which is always integer if we only consider functions $G(x)$ with $G(0)=0$.
- If $A(i_1,\dots,i_k)$ is always an integer multiple of $\binom{k}{j_1,\dots,j_k}$, and $b_n$ is a good sequence, then $c_n = a_n b_n$ is also a good sequence, as $C(i_1,\dots,i_k)$ is an integer:
$$
C(i_1,\dots,i_k)=\frac{A(i_1,\dots,i_k) B(i_1,\dots,i_k)}{\binom{k}{j_1,\dots,j_k}}
$$
For example $c_n = n! 2^{\binom{n}{2}}$ is a good sequence, used to enumerate labeled graphs.
The variant $a_n = x^{\binom{n}{2}}$ is typically used in graphic generating functions, and have a nice combinatorial interpretation when considering direct products $F(z) G(z)$, but considering functional composition for them is uncommon, and exact meaning and interpretation for it remains to be seen, if it exists.
UPD. Generally, this criterion can be interpreted in such a way that we have some kind of baseline combinatorial structure, such that $a_n$ enumerates the number of way to build the structure on $n$ atoms.
For $a_n=n!$, the structure is permutations, while for $a_n = 2^{\binom{n}{2}}$ the structure is undirected graphs or tournaments on $n$ vertices. Then, the wholeness of $A(i_1,\dots,i_k)$ can be interpreted in such a way that by building the base structure on $n$ atoms, we also implicitly and independently from each other build
- A sub-structure of the same kind on first $i_1$ atoms, second $i_2$ atoms, and so on;
- A sub-structure of the same kind on the partition $\{i_1,\dots,i_k\}$ itself.
Then, $A(i_1,\dots,i_k)$ is the number of $n$-structures that are somehow equivalent to each other with regard to the sub-structures.
Then, we use these baseline structures to build an $F$-structure on top of the composition's base structure, and $G$-structures on top of each block of atoms base structure, and interpret whatever we obtained in the end as an $(F \circ G)$-structure on top of $n$ atoms as a whole.
