Let $E_2$, $E_4$, and $E_6$ denote the standard Eisenstein series.
The usual variables $q=e^{2\pi i\tau}$ allow us to regard the
$E_n$'s as functions on either the upper half plane or the unit
disk and we can define $E_n'=\frac{1}{2\pi
i}\frac{d}{d\tau}E_n(\tau)=q\frac{d}{dq}E_n(q)$.   I had cause to
calculate a few of these and saw
$$ E_4'=\frac{1}{3}(-E_6+E_4E_2) $$
$$ E_4''=\frac{5}{36}(E_8-2E_6E_2+E_4E_2^2)$$
$$E_4^{(3)}=\frac{5}{72}(-E_{10}+3E_8E_2-3E_6E_2^2+E_4E_2^3) $$
$$E_4^{(4)}=\frac{35}{864}(E_4^3-4E_{10}E_2+6E_8E_2^2-4E_6E_2^3+E_4E_2^4)-40\Delta $$
and
$$E_6'=\frac{1}{2}(-E_8+E_6E_2) $$
$$E_6''=\frac{7}{24}(E_{10}-2E_8E_2+E_6E_2^2) $$
$$E_4^{(3)}=\frac{7}{36}(-E_4^3+3E_{10}E_2-3E_8E_2^2+E_6E_2^3)+168\Delta $$
It's a standard fact that the derivative of a modular form is
quasimodular, so it's not surprising that we have polynomials in
$E_2$.  I am surprised about the appearance of the binomial
coefficients though.  Is there a deeper reason for their
appearance?  Also, I wonder if the/a pattern continues.  For
instance, it would be interesting if it happens that there always
is some $\alpha \in \mathbb{Q}$ so that
$$E_4^{(n)}-\alpha \sum_{k=0}^{n} (-1)^{k+n}\binom {n}{k}E_{4+2n-2k}E_2^{k}$$
is modular (and similarly for $E_6$).  The other direction you
could ask if the pattern extends is for other modular forms
besides $E_4$ and $E_6$. I've taken a handful of derivatives of
other Eisenstein series and saw similar results.  You don't get
the binomial coefficients though when you take derivatives of
$\Delta$, so maybe at most something general can be said is for
non-cusp forms.