What are some books that discuss elementary mathematical topics ('school mathematics'), like arithmetic, basic non-abstract algebra, plane & solid geometry, trigonometry, etc, in an insightful way? I'm thinking of books like Klein's Elementary Mathematics from an Advanced Standpoint and the books of the Gelfand Correspondence School - school-level books with a university ethos.
Geometry and the Imagination by Hilbert and Cohn-Vossen.
If first-order logic counts as "elementary mathematics", then I would like to suggest (the relevant chapters of) "Godel, Escher, Bach", by Douglas Hofstadter. (As an aside: Hofstadter's puzzle of encoding "n is a power of 10" as a predicate in Peano arithmetic is a wonderful one, quite tough even for professional mathematicians, especially if one is to avoid any form of the Godel numbering trick.)
What Is Mathematics? An Elementary Approach to Ideas and Methods by Richard Courant and Herbert Robbins
Lessons in Geometry by Jacques Hadamard, and its companion books: Hadamard's Plane Geometry and Hadamard: elementary geometry. solutions and notes to supplementary problems by Mark Saul.
Mathematics: A Very Short Introduction by Timothy Gowers. It is very short and indeed very insightful. It is not a textbook, but includes some school-mathematics topics. From the cover:
The aim of this book is to explain, carefully but not technically, the differences between advanced, research-level mathematics, and the sort of mathematics we learn at school.
I really like Concrete Mathematics by Knuth, Graham and Patashnik, and the introductions to number theory by Rose and by Hardy&Wright: you will find there many interesting school-like problems (but the whole books may not be suitable).
In geometry, I can suggest Hartshorne's Geometry: Euclid and beyond.
Books like Géométrie projective by Pierre Samuel or Artin's Geometric algebra contain a lot of algebra, but it is geometric instead of abstract, so you may judge they are on the safe side.
Mathematics Made Difficult by Carl E. Linderholm.
Euclid's elements. i find it much more useful than Klein's books, but that may mean i misunderstand the question. indeed after many years of perusing them, i find Klein's "from an advanced standpoint" books more of a polemic than a useful text. Euclid on the other hand introduces many of the main ideas of modern mathematics.
In "On teaching mathematics", V. Arnold mentions Numbers and Figures by Rademacher and Töplitz, Geometry and the Imagination by Hilbert and Cohn-Vossen, What is Mathematics? by Courant and Robbins, How to Solve It and Mathematics and Plausible Reasoning by Polya, and Development of Mathematics in the 19th Century by F. Klein.
Some of these have been mentioned already, so perhaps this is an appropriate list, but I'm not familiar with all of these, so if someone would like to comment on these books, your input would be appreciated.
It was ages ago that I read in a library Mathematics: Its Content, Methods, and Meaning by Aleksandrov, Kolmogorov, and Lavrent'ev, but I still remember enjoying it.
How about Lawvere&Schanuel's "Conceptual Mathematics: A First Introduction to Categories"? See, e.g., http://books.google.fr/books?id=h0zOGPlFmcQC&lpg=PP1&dq=lawvere&pg=PP1#v=onepage&q=lawvere&f=false
While it contains much beyond school mathematics, a lot of school mathematics is treated in a beautiful way in Mathematics, Form and Function by Saunders Mac Lane.
Mathematics The Science of Patterns by Keith Devlin.
I would suggest "Numbers and functions from a classical-experimental mathematicians point of view" by Victor H.Moll. It contains very elementary but also some more sophisticated themes.
I once also wrote an elementary book "Grundideen der Mathematik", B.I. Wissenschaftsverlag 1992. But it is out of print and in German, thus probably does not count.
Stewart's book is an old favorite:
Stewart, Ian. From here to infinity. With a foreword by James Joseph Sylvester. The Clarendon Press, Oxford University Press, New York, 1996.
Stewart has many popularisation books some of which have reached best-seller status, e.g.: