In Hamilton's article on the Nash-Moser Theorem, he gives the map that maps a vector field $X$ to its flow $e^{tX}$ in $\mathrm{Diff}(M)$ as an example where the implicit function theorem in Frechet spaces fails: The derivative of this map at the zero vector field is the identity, yet the map is not surjective in any neighborhood of the zero section in the space of vector fields.
However, when thinking about this example, I realized that I was not sure how to construct the manifold structure on $\mathrm{Diff}(M)$. The first idea would be to choose a Riemannian metric on $M$ and define a map from vector fields to $\mathrm{Diff}(M)$, $X \mapsto \phi_X$ by setting $$ \phi_X(p) := \exp_p(X(p))$$ using the Riemannian exponential map. The claim would then be that this map is a diffeomorphism on some neighborhood of the zero section.
However, it seems to me that again, one would use an Implicit function type of statement to make this conlusion.
So why does the latter work and the first doesn't? Or rather, the map $X \mapsto e^{tX}|_{t=1}$ is certainly injective for $X$ small enough (right?) so one could just use this map to define the manifold structure in the first place (or why does this fail?).