Wolfram MathWorld says
As used in physics, the term “exact” generally refers to a solution that captures the entire physics and mathematics of a problem as opposed to one that is approximate, perturbative, etc. Exact solutions therefore need not be closed-form.
But Terry Tao seems to take "exact solution" to mean a closed form solution.
Is one of these very standard? Or does usage just vary on this point?
It depends on context. In the physics literature, there is a term "exactly solvable" meaning that a closed form for the solution can be written; it is never used to indicate that the solution exists in an abstract sense. E. g., see Baxter's classical book "Exactly solvable models in Statistical mechanics". So, in this context "exact solution" does mean "closed form solution".
Edit inspired by Alexandre Eremenko's answer: more precisely, for a physicist, "exact solution" should be explicit enough to answer all the questions they care about. In practice, these questions often boil down to the asymptotics at singular points and/or infinity. Say, Baxter's exact solutions are primarily interisting because they provide the critical exponents. In that sense, Painleve functions should count as closed form, since the asymptotical expansions and connection formulae are known for them, and Sundman's solution is not.
In other context, you may talk about approximate or perturbative solution, and then I feel it is fine to contrast it to the "exact solution" even when the closed form for the latter is not known.
There is no formal definition. This depends on context.
Those who say that "exact solution" means a "closed form solution" have to explain what a "closed form" is. A series whose coefficients are rational numbers, and there is an algorithm to compute these coefficients, is this a closed form or not? Some of them are. Everyone would say that solution $y(x)=e^x$ of a differential equation is a "closed form". Then all other special functions must be admitted (like Bessel, elliptic functions, or functions of elliptic cylinder). Now, what is a precise definition of "special functions"? The meaning of this term changes with time. Most people agree that those functions considered in the second volume of Whittaker Watson deserve this name. What about solutions of Heun equations (this equation is only mentioned once in WW, in an exercise)? What about Painleve functions (not mentioned in WW)? What about this series with rational coefficients: $$f_q(z)=\sum_{m=0}^\infty\frac{q^{-n^2}z^n}{n!}, \quad z\in C, \quad |q|\leq 1.$$ If you obtain a solution of a problem in this form, will this be counted a closed form solution? Exact solution?
In 19th century, they proposed this definition: an explicit solution is a series whose coefficients can be computed (say, by finite recurrent formula), and which converges for all values of the independent variable relevant for the given question. Then, shortly after that, Sundman solved the famous three body problem in this form. Still most people today will not call Sundman's solution a closed form or explicit solution.
Sometimes they mean "an elementary function" as a closed form, though there is no agreement what exactly an "elementary function" is. What about an Abelian integral, is this a closed form solution or not? For most physicists, it is. On the other hand the study of qualitative behavior of Abelian integrals is a hot research area nowadays.
Ref. Here is how the prize problem was formulated for the many bodies problem:
For a system of arbitrarily many mass points that attract each other according to Newton’s law, assuming that no two points ever collide, find a series expansion of the coordinates of each point in known functions of time converging uniformly for any period of time.
The prize was awarded to Poincare for his groundbreaking work, though he did not solve the problem, as stated. This was achieved, for 3 bodies, few years later by Sundman. A modern exposition of Sundman's solution can be found in the book by Siegel, Celestial mechanics. See also this MO answer. Poincare work started a new area of research which flourishes nowadays, while Sundman's work is almost forgotten.
There is no "exact answer" to this question. Answers will contain the words "reasonable" and "appropriate", terms which depend on the context. I'll try to give a reasonable answer.
Given a domain of parameters for which a problem is defined, an "exact" solution is one that permits one to compute an answer to any reasonable desired accuracy for the whole domain with a reasonable amount of effort.
"Reasonable" accuracy is defined by the physical question - typically, whether it makes sense to define the quantity in question beyond a certain accuracy, or whether it is feasible to measure the quantity in question beyond a certain accuracy in an experiment. It makes no sense to define the distance between two cities beyond, say, the size of a typical city hall. It is futile to make a prediction for an experiment beyond the accuracy achievable in any conceivable experiment within, say, decades.
"Reasonable" effort depends in part on the importance of the problem, and again on time scales. If it takes longer than, say, decades to compute the answer, that's certainly unreasonable. The computational resources committed to the problem must be commensurate with the importance of the problem as established by societal consensus, otherwise the effort is again unreasonable.
An additional qualifier often used is "in principle". "In principle exact" means that one has a solution that yields any reasonable accuracy, but that the effort involved would be unreasonable. There is the quip that Feynman reduced all of physics to quadrature by coming up with the path integral (well - caveat - to the extent that one knows the action); this solution might be seen as in principle exact, but it's not truly an "exact solution".
