Complex Multiplication and algebraic integers Let $q=e^{2\pi i\tau}$ and 
$$E_2(\tau) = 1 - 24 \sum_{n=1}^\infty\frac{nq^n}{1-q^n}$$ be the Eisenstein Series of weight $2$
and let $E_2^*(\tau) = E_2(\tau) - \frac{3}{\pi\cdot Im(\tau)}$ be the corresponding almost holomorphic modular form. Then let
$$\eta(\tau)=e^{\pi i\tau /12}\cdot\prod_{n=1}^\infty(1-q^n)$$
be the Dedekind $\eta$-Function.
Question: How can I prove the following statement from here:

Let $\tau$ be any Complex Multiplication point. By basic theorems of complex multiplication, if you choose a suitable period
  $\omega(\tau)$, $E_4(\tau)/\omega(\tau)^4$, $E_6(\tau)/\omega(\tau)^6$,
  and $\sqrt{D}E_2^*(\tau)/\omega(\tau)^2$ (with $E_2^*(\tau)=E_2(\tau)-3/(\pi \cdot Im(\tau))$ and $D$ the discriminant of $\tau$) will be algebraic numbers of known degree, and if you choose
  $\omega(\tau)=\eta(\tau)^2$, they will even be algebraic integers.

Partial Solution for $E_2^*$:
Francois Brunault pointed out that this statement can be found in Prop. 5.10.6 on p. 202 of Cohen/Strömberg's book Modular Forms: A Classical Approach.
Unfortunately, the proof of this proposition starts with "we only prove algebraicity, not the integrality". Who can help with proving the integrality? Since I am no expert in complex multiplication, I am looking for a rather detailed answer, or for a reference.
Complete Solution for $E_4$ and $E_6$:
The statement that $\frac{E_4}{\eta^8}$ is an algebraic integer follows from
$$\left(\frac{E_4}{\eta^8}\right)^3 = j(\tau)$$
and the statement that $\frac{E_6}{\eta^{12}}$ is an algebraic integer follows from
$$\left(\frac{E_6}{\eta^{12}}\right)^2 = j(\tau)-1728$$
with the absolute invariant $j(\tau)=\frac{1728E_4^3}{E_4^3-E_6^2}=\frac{E_4^3}{\eta^{24}}$ which is an algebraic integer (see Silverman, Advanced Topics in the Arithmetic of Elliptic Curves, Theorem 6.1, p. 140).
EDIT: Complete Solution for $E_2^*$:
The answer of Michael Griffin (see below) can now be found in more details in the appendix of this arXiv-preprint.
 A: I have seen this statement about $E_2^*$ tossed around off-handedly by experts a number of times, but never seen a complete proof referenced.
The tools to prove it are (mostly) in Masser's "Elliptic functions and transcendence", Appendix 1. There, Masser gives a formula for certain non-holomorphic modular functions. One of these fomrulas (Lemma A3) can be re-written as 
$$E_2^*(\tau)\left(\frac{\pi}{\omega_1}\right)^2=-\frac{3S}{\sqrt{D} ~\tau},$$ where $(\omega_1,\omega_2)$ are choices of periods of a CM elliptic curve with rational equation, $D$ is the discriminant of the CM point $\tau=\frac{\omega_1}{\omega_2}$, and $S$ is a sum of division points on the curve. If $\tau$ satisfies the reduced, integral quadratic, $C\tau^2+B\tau+A$, then Masser points out that by a theorem of Baker, $(AC)^2\wp$ is an algebraic integer. Moreover, the norm of $\tau$ is $A/C$, and so it's clear the only additional primes that could divide the denominator are divisors of $AC$. 
On page 118 of Masser, he offers formulas for the function 
$$\gamma(\tau)=\frac{E_2^*(\tau)E_4(\tau)}{6E_6(\tau) j(\tau)}-\frac{7j(\tau)-6912}{6j(\tau)(j(\tau)-1728)}$$
at CM points, in terms of singular moduli of $j(\tau)$. The Gross-Zagier formula (which gives a factorization for the norm of differences of singular moduli) can then be used to show that no primes that split in the CM field can divide the denominator. Any prime dividing $A$ or $C$ is either split or ramified in the CM field. None of the split primes can divide the denominators. A more careful use of the Gross-Zagier formula shows that the ramified primes appear no more than expected, and so $\sqrt{D}E_2^*(\tau)\left(\frac{\pi}{\omega_1}\right)^2$ must be an algebraic integer.
A: The statement about $E_2^*(\tau)$ is Proposition 5.10.6 in Cohen-Strömberg, Modular forms: a classical approach.
