What books collect examples and counterexamples (or also "solved exercises", for some suitable definition of "exercise") in
The only book of this kind that I know of is the famous
Counterexamples in Analysis by Bernard R. Gelbaum and John M. H. Olmsted.
I coincidentally read this question, and despite it is not currently active, I believe it is right to share with the forum my personal answer. I think that reference 1 is very nice book for what pertains to the last two points of your question. It is not a list of simple exercises like other books on the same topics, but a collection of many carefully chosen problems (many of them at the research level) and examples illustrating many more aspects of ODE (in the first part of the book) and PDE theory (in the second part), with answer or hints for solution. For example you can find the Garabedian-Grushin and the Lewy examples, examples in semigroup theory and in the theory of functions of several complex variables, examples pertaining the Gilbarg-Serrin theorem (you may notice that I am more accustomed to the second part of the book). In order to get a more precise idea of the aim of the book see the main portion of the Author's preface below and also its Zbl review.
1 Biler, P., Nadzieja, T. Problems and Examples in Differential Equations, Pure and Applied Mathematics 164, New York:Marcel Dekker, Inc., ISBN: 0-8247-8637-8, viii+244 (1992), MR1198886, Zbl 0760.34001.
In the book
Khaleelulla S. Counterexamples in Topological Vector Spaces, LNM 936, Springer, 1982
various counterexamples in Topological Vector Spaces, Locally Convex Spaces, Ordered Topological Vector Spaces, and Topological Algebras are given.
In the following reference (in French however), many interesting counter-examples (not only in analysis) are constructed.
Bertrand Hauchecorne, Les contre-exemples en mathématiques, second edition, Ellipses, 2007.
Otherwise, I think that Bourbaki is always a good source of counter-examples, even if its exercises are not corrected, thanks of the numerous dangerous bend symbols throughout the different volumes.
