Good "casual" advanced math books I'm curious if there are any good math books out there that take a "casual approach" to higher level topics. I'm very interested in advanced math, but have lost the time as of late to study textbooks rigorously, and I find them too dense to parse casually.
By "casual", I mean something that goes over maybe the history of a certain field and its implications in math and society, going over how it grew and what important contributions occurred at different points. Perhaps even going over the abstract meaning of famous results in the field, or high level overviews of proofs and their innovations. Conversations between mathematicians at the time, stories about how proofs came to be, etc.
I suppose the best place to start would be math history books, but I was curious what else there may be. Are there any good books out there like this? What are your recommendations?
 A: Birth of a Theorem: A Mathematical Adventure, by Cedric Villani. I've linked to a review. 
Also: Diaconis & Skyrms, Ten Great Ideas About Chance. I quote from the Preface: 
This is a history book, a probability book, and a philosophy book. We give the history of what we see as great ideas in the development of probability, but we also pursue the philosophical import of these ideas....
At the beginning of this book we are thinking along with the pioneers, and the tools involved are simple. By the end, we are up to the present, and some technicalities have to be at arms length. We try to ease the flow of exposition by putting some details in appendices, which you can consult as you wish. We also try to provide ample resources for the reader who finds something interesting enough to dig deeper. 
A: I would add 3 favorites:
The first two (more or less continuing one another) are by W Narkiewicz and are quite comprehensive combining history, some proofs, excellent references and state of the art results up to ~2010:
The Development of Prime Number Theory : From Euclid to Hardy and Littlewood 
Rational Number Theory in the 20th Century: From PNT to FLT
Finally a superb exposition with proofs, explanations, history, from D Choimet and H Queffelec:
Twelve Landmarks of Twentieth-Century Analysis
A: In 'How to bake a π?', Eugenia Cheng provides a nice view of mathematics in general and category theory in particular that may fit your definition of 'casual'. Either way it is a very nice reading. 
A: What's one person's "higher level topics" is another person's "elementary math", so you should be more specific about the desired level.
But still you may try these books:

*

*Michio Kuga, Galois' dream,


*David Mumford, Caroline Series, David Wright, Indra's Pearls,


*Hermann Weyl, Symmetry.


*Marcel Berger, Geometry revealed,


*D. Hilbert and Cohn-Vossen, Geometry and imagination,


*T. W. Körner, Fourier Analysis,


*T. W. Körner, The pleasures of counting.


*A. A. Kirillov, What are numbers?


*V. Arnold, Huygens and Barrow, Newton and Hooke.


*Mark Levi, Classical mechanics with Calculus of variations
and optimal control.


*Shlomo Sternberg, Group theory and physics,


*Shlomo Sternberg, Celestial mechanics.
All these books are written in a leisurely informal style, with a lot of
side remarks and historical comments, and almost no prerequisites. But the level of sophistication varies widely. Also don't miss:
Roger Penrose, The road to reality. A complete guide to the laws of the universe. It is on physics, but contains a lot of mathematics.
A: Manfred Schroeder's book entitled Number Theory in Science and Communication: With Applications in Cryptography, Physics, Digital Information, Computing, and Self-Similarity is a wonderful introduction to applied discrete mathematics, or concrete mathematics, to use Knuth's phrase.
https://www.springer.com/gp/book/9783540852971
"A light-hearted and readable volume with a wide range of applications to which the author has been a productive contributor – useful mathematics outside the formalities of theorem and proof." Martin Gardner
A: I enjoyed Allen Hatcher's "Algebraic Topology" very much.
It's free. You can find it online as a PDF on his university's webspace: https://pi.math.cornell.edu/~hatcher/AT/AT.pdf
Hope it will stay up there for long. He's been retired for a while now.
That was maybe the most enjoyable math book I've read.
A: "Moonshine beyond the Monster" by Terry Gannon. I bought this book on a whim and then I couldn't put it down. Very good elementary overview of some deep mathematics.
A: Silvanus P. Thompson's Calculus Made Easy is a good read. Per the reviews, the paperback and Kindle versions should be avoided in favor of the hardcover, although my softcover copy predates this by 2-3 decades.
A: Elementary Applied Topology by Ghrist does a fantastic job surveying recent trends in the application of (co)homological methods to practical science and engineering. It goes all the way from Euler characteristic to sheaf cohomology.
Oh, and the illustrations, depending on how far you get with deciphering them, are pretty cool too:

A: For a casual and beautiful promenade, there is A Singular Mathematical Promenade by E. Ghys.
For French readers, the collection
Leçons de mathématiques d'aujourd'hui (there are now five volumes) presents a panorama of various subjects and research domains in (mostly pure) mathematics accessible to graduate or advanced undergraduate students.
A: I think “Proofs from THE BOOK” of Aigner and Ziegler may be of some interest. Although the book is not primarily historical, it contains such aspects as well. The book is casual. I do not know if it is advanced or too elementary for you.  
A: Computing the Continuous Discretely by Beck and Robins.
Good intro to the interplay of analysis (Fourier analysis and number theory), geometry, and combinatorics.
Google books, pdf
Chapter 10: Topology grows into a branch of mathematics  in Never a Dull Moment: Hassler Whitney, Mathematics Pioneer by Keith Kendig**
Zeros of Entire Fourier Transforms by
Dimitar Dimitrov and Peter Rusev
A long paper/short book on identifying polynomials and entire functions that have only real zeros and the influence of and applications to the Riemann hypothesis.
Learning Modern Algebra From Early Attempts to Prove Fermat's Theorem by Cuoco and Rotman.
Möbius and his Band: Mathematics and Astronomy in Nineteenth-century Germany edited by Fauvel, Flood, and Wilson.
A compilation of articles:

*

*A Saxon mathematician by John Fauvel


*The German mathematical community by Gert Schubering


*The astronomical revolution by Allan Chapman


*Möbius's geometrical mechanics by Jeremy Gray


*The development of topology by Norman Biggs


*Möbius's modern legacy by Ian Stewart
I had known of Möbius chiefly through the Listing-Möbius band, linear fractional transformations, and the Möbius function and inversion--all of continuing significance in modern mathematics. The articles cover more and provide a nice entre/appetizer for modern topics.
A: I like the books in the Student Mathematical Library that are published by the AMS. 
I am currently reading "Modern Cryptography and Elliptic Curves, A Beginner's Guide" by
Thomas Shemanske. It is quite delightful.
A: I would add in the recent book Change and Variations by Jeremy Grey. It provides a beautiful historical overview of the evolution from the beginnings of differential equations and calculus of variations to the modern theory. Though it’s written with a historical bent, it’s definitely rigorous and could easily be used for a course.
A: I very much liked a book by David A Cox titled Galois Theory. The author writes in the section preface to the first edition

This book was written in an attempt to do justice to both the history and the power of Galois theory. My goal for students to appreciate the elegance of the theory and simultaneously have a strong sense of where it came from.

Happy reading!!
A: Based on your first paragraph, I would highly recommend the series What's Happening in the Mathematical Sciences.  These provide excellent summaries of a wide variety of cutting-edge mathematics topics.  The authors are mathematicians and so the accuracy of the discussion is very high and there is enough detail to satisfy the casual interest of a mathematician, but they also don't get bogged down in too much detail.
There have even been times when I wanted to gain a thorough understanding of a new and unfamiliar topic, and I found that the introduction in What's Happening was better than any other introduction I could find.  Of course I then I had to turn to more technical texts for more detail, but the overall perspective provided by the What's Happening article was invaluable.
A: I think that the "Number Theory" series by Kato, Kurokawa, Kurihara and Saito fits here. They are beautifully written and require only some undergradute algebra and analysis. They were published (translated) in the AMS Translations of Mathematical Monographs:
Number theory. 1. Fermat's dream
Number theory. 2. Introduction to class field theory
Number theory. 3. Iwasawa theory and modular forms
A: Regarding differential geometry and topology, there is the 3 volume "A Mathematical Gift - The Interplay Between Topology, Functions, Geometry, and Algebra" by K. Ueno, K. Shiga, S. Morita, T. Sunada. The level is "relaxed undergraduate mathematics". The book attempts to bring to the front the intuition behind some of the concepts encountered in geometry. (All the 4 authors are mathematicians working in Japanese universities.)
A: For a casual book on cryptography and the mathematics behind it, I'll recommend Simon Singh's 'The Code Book'. The concept of modular mathematics and public key cryptography are explained beautifully.  
A: I highly recommend "Office Hours With a Geometric Group Theorist", edited by Matt Clay and Dan Margalit, which is a series of essays on various topics in Geometric Group Theory written in a very informal style.
A: I personally enjoyed The KAM Story by H. Scott Dumas. It gives an overview of the history of KAM theory. Very enjoyable and yet informative.
A: James Gleick - Chaos: Making a New Science is a popular history I still remember from 25 years ago.
A: I find the Carus Mathematical Monographs to be in this category.
Also, by Julian Havil - "The Irrationals" and "Exploring Gamma".
A: I'm surprised that no one has yet mentioned The Symmetries of Things by John Conway, Heidi Burgiel and Chaim Goodman-Strauss. It's ostensibly an introduction to symmetry groups but touches into classifying hyperbolic tilings, the number of groups of a given order, 'infinite' polyhedra and several other fascinating related topics. It's casually enough written and well enough illustrated that it's easy to underestimate how deep you've gone until you take a look around.
A: This is borderline advanced mathematics, but my impression is that

*

*Andrew Granville, Number Theory Revealed: A Masterclass, 2020
could well be used this way. Many topics, some rather obscure, are covered, but the technical prerequisites are never particularly forbidding, nor is there much buildup; it's as bite-sized as a textbook can get.
Another field that has recently attracted semi-popular treatments are $p$-adic numbers. Here are two that come to my mind:

*

*Svetlana Katok, p-adic Analysis Compared with Real, AMS 2007. (Errata.)


*Fernando Gouvea, p-adic Numbers: An Introduction, 3rd edition, Springer 2020.
Finally,

*

*Titu Andreescu, Gabriel Dospinescu, Problems from the Book, XYZ Press, 2nd edition 2010;


*Titu Andreescu, Gabriel Dospinescu, Straight from the Book, XYZ Press, 2012
are meant as olympiad training texts but can be read profitably as light reasoning (avoiding the problems), as they contain a lot more theoretical discussion than the typical olympiad text (and both the theory and the problems are chosen with a lot of taste).
A: Since no one has mentioned this I found the following trilogy by Ash and Gross fits your description: Fearless Symmetry , Elliptic Tales and Summing It Up.
I personally had enjoyed reading them and found the semi-technical but mostly conceptual writing to be truly enlightening in shaping my understanding of the maths fields in question when I had no formal training in them.  In fact after reading them I kinda knowing how to approach themes of the books with further technicalities despite not being committed to do so.
