As is well known, in the typical proof of the Picard–Lindelöf theorem, one shows the existence of a solution of the initial value problem $y'(t) = f(t,y(t))$, $y(t_0) = y_0$ by considering the Picard iteration, $$ \varphi_0(t) = y_0, $$ $$ \varphi_{k+1}(t) = y_0 + \int_{t_0}^t f(s,\varphi_k(s))ds, $$ which ideally converges to a solution of the initial value problem in some neighborhood of $t_0$.
This is (as far as I know) a purely theoretical tool, as these integrals quickly become numerically and algebraically infeasible. Nevertheless, suppose that a particularly diligent high school calculus student (who is well versed in the differential algebra literature and has memorized all 100+ pages of the specification of the Risch algorithm) sees this definition and tries to implement it by hand on some initial value problem. How far will they get?
Clearly if $f(t,y)$ is just a polynomial, the Picard iterates will all be polynomials too, but if $f(t,y)$ is some more complicated function it becomes uncertain fairly quickly. If we just consider $y'(t) = e^y$, $y(0) = 0$, for instance, we get $\varphi_0(t) = 0$, $\varphi_1(t) = t$, $\varphi_2(t) = e^t-1$, and then $\varphi_3(t) = \int_0^t e^{e^s-1}ds = \frac{1}{e}(\mathrm{Ei}(e^t) - \mathrm{Ei}(1))$, which is not elementary.
Existence of elementary antiderivatives is also famously delicate in general. Wikipedia gives an example of the functions $\frac{x}{\sqrt{x^4+10x^2 - 96x - 71}}$, which has an elementary antiderivative, and $\frac{x}{\sqrt{x^4+10x^2 - 96x - 72}}$, which does not. Given this it's plausible that in some sense the only class of functions for which all Picard iterates are elementary 'generically' is the polynomials, although I'm not sure how to formalize that idea exactly.
Recall that an elementary function is a member of the smallest set of analytic functions containing polynomials over $\mathbb{C}$, $e^x$, and $\log x$ and closed under addition, subtraction, multiplication, division, and composition.
We can assume without loss of generality that $t_0 = 0$ and $y_0 = 0$, so given an elementary function $f(t,y)$ which is analytic in some neighborhood of $(0,0)$, let $\varphi_0^{f}(t) = 0$ and $\varphi^{f}_{k+1}(t) = \int_0^t f(s,\varphi^{f}_k(s))ds$ for all $k$.
Question 1. For which $f$ is $\varphi_k^{f}(t)$ an elementary function for all $k$?
Question 2. For which $n$ does there exist an $f$ such that $\varphi_k^{f}(t)$ is elementary for all $k < n$ but $\varphi_n^{f}(t)$ is not?
