I have seen this statement about $E_2^*$ tossed around off-handedly by experts a number of times, but never seen a complete proof referenced.

The tools to prove it are (mostly) in Masser's "Elliptic functions and transcendence", Appendix 1. There, Masser gives a formula for certain non-holomorphic modular functions. One of these fomrulas (Lemma A3) can be re-written as $$E_2^*(\tau)\left(\frac{\pi}{\omega_1}\right)^2=-\frac{3S}{\sqrt{D} ~\tau},$$ where $(\omega_1,\omega_2)$ are choices of periods of a CM elliptic curve with rational equation, $D$ is the discriminant of the CM point $\tau=\frac{\omega_1}{\omega_2}$, and $S$ is a sum of division points on the curve. If $\tau$ satisfies the reduced, integral quadratic, $C\tau^2+B\tau+A$, then Masser points out that by a theorem of Baker, $(AC)^2\wp$ is an algebraic integer. Moreover, the norm of $\tau$ is $A/C$, and so it's clear the only additional primes that could divide the denominator are divisors of $AC$.

On page 118 of Masser, he offers formulas for the function $$\gamma(\tau)=\frac{E_2^*(\tau)E_4(\tau)}{6E_6(\tau) j(\tau)}-\frac{7j(\tau)-6912}{6j(\tau)(j(\tau)-1728)}$$ at CM points, in terms of singular moduli of $j(\tau)$. The Gross-Zagier formula can then be used to show that no primes that split in the CM field can divide the denominator. Since any prime dividing $A$ or $C$ must split in the CM field, none of these primes can divide the denominators, and so $\sqrt{D}E_2^*(\tau)\left(\frac{\pi}{\omega_1}\right)^2$ must be an algebraic integer.

I have seen this statement about $E_2^*$ tossed around off-handedly by experts a number of times, but never seen a complete proof referenced.

The tools to prove it are (mostly) in Masser's "Elliptic functions and transcendence", Appendix 1. There, Masser gives a formula for certain non-holomorphic modular functions. One of these fomrulas (Lemma A3) can be re-written as $$E_2^*(\tau)\left(\frac{\pi}{\omega_1}\right)^2=-\frac{3S}{\sqrt{D} ~\tau},$$ where $(\omega_1,\omega_2)$ are choices of periods of a CM elliptic curve with rational equation, $D$ is the discriminant of the CM point $\tau=\frac{\omega_1}{\omega_2}$, and $S$ is a sum of division points on the curve. If $\tau$ satisfies the reduced, integral quadratic, $C\tau^2+B\tau+A$, then Masser points out that by a theorem of Baker, $(AC)^2\wp$ is an algebraic integer. Moreover, the norm of $\tau$ is $A/C$, and so it's clear the only additional primes that could divide the denominator are divisors of $AC$.

On page 118 of Masser, he offers formulas for the function $$\gamma(\tau)=\frac{E_2^*(\tau)E_4(\tau)}{6E_6(\tau) j(\tau)}-\frac{7j(\tau)-6912}{6j(\tau)(j(\tau)-1728)}$$ at CM points, in terms of singular moduli of $j(\tau)$. The Gross-Zagier formula (which gives a factorization for the norm of differences of singular moduli) can then be used to show that no primes that split in the CM field can divide the denominator. Any prime dividing $A$ or $C$ is either split or ramified in the CM field. None of the split primes can divide the denominators. A more careful use of the Gross-Zagier formula shows that the ramified primes appear no more than expected, and so $\sqrt{D}E_2^*(\tau)\left(\frac{\pi}{\omega_1}\right)^2$ must be an algebraic integer.