I may start answering by pointing out that the term "nonparametrics statistics" is essentially "parametric". The existing methods (e.g. Smoothing splines) in nonparametrics, are somehow all parametrized by some (finite dimensional) set of parameters.
The term "flexible" is true. However, from a applied perspective, you need to conduct a model selection to choose a fixed parameter space to do statistical inference. Alternatively, if you go for a Bayesian nonparametric modeling, instead of model selection, people usually will do model averaging (e.g. RJ-MCMC used for Bayesian modeling)
...there will be too much freedom and hence the problem will become too easy (a piecewise linear function joining the points being a trivial solution), while I think it is safe to assume that such a solution will likely shed no light on the phenomenon under study...
Again, let's consider the smoothing splines. The main restriction we attempt to impose is "knot conditions" that lead to certain order of smoothness at certain sub-domains. Once you put these restraints and choose the splines as basis, the linearity arise from the space of these basis functions. Therefore, "parametric linear" statistical inference like ANOVA can again be conducted. The answer to your question can also be found in the same book.
In a more general sense, you can do some probabilistic inference without assuming linearity, for example, in a Banach space (e.g. Probability in Banach Space). But in that case, most object you derive will not have specific forms like splines.