Essays and thoughts on mathematics 
Many distinguished mathematicians, at some point of their career,
  collected their thoughts on mathematics (its aesthetic, purposes,
  methods, etc.) and on the work of a mathematician in written form.

For instance:


*

*W. Thurston wrote the lovely essay On proof and progress in mathematics in response to an article by Jaffe and Quinn; some points made there are also presented in an answer given on MathOverflow (What's a mathematician to do?). 

*More recently, T. Tao shared some personal thoughts and opinions on what makes "good quality mathematics" in What is good mathematics?.

*G. Hardy wrote the famous little book A Mathematician's Apology, which influenced, at least to some extent, several generations of mathematicians.


Personally, I've been greatly inspired by the two writings listed under (1.) --  they are one of the main reasons why I started studying mathematics -- and, considering that one of them appeared on MathOverflow, I'd like to propose here -- if it is appropriate -- to create a "big-list" of the kind of works described in the above blockquote. 

I'd suggest (again, if it is appropriate) to give one title (or link) per answer with a short summary.



*

*A related question, which I've found very interesting, is 
Good papers/books/essays about the thought process behind mathematical research.

*Only slightly related (but surely interesting): Which mathematicians have influenced you the most?

*A single paper everyone should read? is not quite related, but still somewhat relevant (especially the most up-voted answer).

 A: Eugene Wigner: The Unreasonable Effectiveness of Mathematics in the Natural Sciences 

The statement that the laws of nature are written in the language of
  mathematics was probably made three hundred years ago [It is
  attributed to Galileo]. It is now more true than ever before …  Surely
  complex numbers are far from natural or simple and they cannot be
  suggested by physical observations. Furthermore, the use of complex
  numbers is close to being a necessity in the formulation of the laws
  of quantum mechanics. It is difficult to avoid the impression that a
  miracle confronts us here, quite comparable in its striking nature
  to the miracle that the human mind can string a thousand arguments
  together without getting itself into contradictions, or to the two
  miracles of the existence of laws of nature and of the human mind’s
  capacity to divine them. The closest explanation [for this
  mathematical universe] is Einstein’s statement that “the only physical
  theories which we are willing to accept are the beautiful ones”  … the
  concepts of mathematics have this quality of beauty.

A: A Drifter of Dadaist Persuasion by Matilde Marcolli, published in Art in the Life of Mathematicians (Edited by Anna Kepes Szemerédi) American Mathematical Society, 2015, pp.210-231
A: The Psychology of Invention in the Mathematical Field (Jacques Hadamard's 1945 essay)
A: The Mathematician by John Von Neumannn.
A: Enigmas of Chance, by Mark Kac.
A: I would add "Letters to a Young Mathematician" by Ian Stewart
A: I recommend:

Vladimir Arnold: Yesterday and Long Ago. This is a very enjoyable and highly interesting collection of anecdotes and historical remarks. The latest Russian edition of this book contains some more chapters.
Richard Hamming: You and Your Research, transcribed and edited by J F Kaiser, reprinted in Tveito et al: Simula Research Laboratory. This is the text of a lecture of Hamming.

A: Birth of a Theorem, by French candidate for Parliament Cédric Villani
A: There are many snippets that can be found. I like the following bit of the foreword by Thurston to J. H. Hubbard's Teichmüller Theory. I share the remarks because I think you simply can't have enough of Bill Thurston's insights: 

"Mathematics is a paradoxical, elusive subject, with the habit of
  appearing clear and straightforward, then zooming away and leaving us
  stranded in a blank haze.
Why?
It is easy to forget that mathematics is primarily a tool for human
  thought. Mathematical thought is far better defined and far more
  logical than everyday thought, and people can be fooled into thinking
  of mathematics as logical, formal, symbolic reasoning. But this is far
  from reality. Logic, formalization, and symbols can be very powerful
  tools for humans to use, but we are actually very poor at purely
  formal reasoning; computers are far better at formal computation and
  formal reasoning, but humans are far better mathematicians.
The most important thing about mathematics is how it resides in the
  human brain. Mathematics is not something we sense directly: it lives
  in our imagination and we sense it only indirectly. The choices of how
  it flows in our brains are not standard and automatic, and can be very
  sensitive to cues and context. Our minds depend on many interconnected
  special-purpose but powerful modules. We allocate everyday tasks to
  these various modules instinctively and subconsciously.
The term `geometry', for instance, refers to a pattern of processing
  within our brains related to our spatial and visual senses, more than
  it refers to a separate content area of mathematics. One illustration
  of this is the concept of correlation between two measurements on a
  set, which is formally nearly identical with the concept of cosine of
  the angle between two vectors. The content is almost the same (for
  correlation, you first project to a hyperplane before measuring the
  cosine of the angle), but the human psychology is very different. Each
  mode of thinking has its own power, and ideally, people harness both
  modes of thought to work together. However, in formalized expositions,
  this psychological > difference vanishes.
In the same way, any idea in mathematics can be thought about in many
  different ways, with competing advantages. When mathematics is
  explained, formalized and written down, there is a strong tendency to
  favor symbolic modes of thought at the expense of everything else,
  because symbols are easier to write and more standardized than other
  modes of reasoning. But when mathematics loses its connection to our
  minds, it dissolves into a haze.
I've loved to read all my life. I went to New College of Sarasota,
  Florida, a small college that was just starting up with a strong
  emphasis on independent study, so I ended up learning a good deal of
  mathematics by reading mathematics books. At that time, I prided
  myself in reading quickly. I was really amazed by my first encounters
  with serious mathematics textbooks. I was very interested and
  impressed by the quality of the reasoning, but it was quite hard to
  stay alert and focused. After a few experiences of reading a few pages
  only to discover that I really had no idea what I'd just read, I
  learned to drink lots of coffee, slow way down, and accept that I
  needed to read these books at 1/10th or 1/50th standard reading speed,
  pay attention to every single word and backtrack to look up all the
  obscure numbers of equations and theorems in order to follow the
  arguments. Even so, when something was ``left to the reader'', I
  generally left it as well. At the time, I could appreciate that the
  mathematics was an impressive intellectual edifice, and I could follow
  the steps of proofs. I assumed that such an elaborate buildup must be
  leading to a fantastic denouement, which I eagerly awaited -- and
  waited, and waited.
It was only much later, after much of the mathematics I had studied
  had come alive for me that I came to appreciate how ineffective and
  denatured the standard ((definition theorem proof)^n remark)^m style
  is for communicating mathematics. When I reread some of these early
  texts, I was stunned by how well their formalism and indirection hid
  the motivation, the intuition and the multiple ways to think about
  their subjects: they were unwelcoming to the full human mind.
John Hubbard approaches mathematics with his whole mind.
If you page through the current book, you will see many intriguing
  figures. That is a first sign: figures are one of the most important
  ways to keep our thought processes going in our whole brains, rather
  than settling down into the linguistic, symbol-handling areas. Of
  course, the figures in your imagination are even more important.
  Geometric ideas can be conveyed with words and with symbols, sometimes
  more effectively than with pictures, but a lack of figures is a good
  indication of a lack of geometry.
Another important part of human thinking is the emotional aspect. In
  mathematics, what is intriguing, puzzling, interesting, surprising,
  boring, tedious, exciting is crucial; they are not incidental, they
  shape how we think. Personally, my thinking was shaped by boredom: I
  develop intense urges to come up with `easy' methods in order to avoid
  tedious computations that are opaque to me. Hubbard, a principal
  participant in the mathematics he is discussing, has done an excellent
  job in conveying the drama."

There are also many very good interviews that can be found, such as this one with Connes, as well as the advice to young mathematicians in the Princeton Companion to Mathematics. 
A: A Mathematician's lament by Paul Lockhart: Reflections on how badly mathematics are taught these days. Imagining how it would be if music was taught the same way.
A: Here are additional mathematicians' thoughts.
S. Ulam, Adventures of a mathematician.A recollection of his life, from Lwow to Los Alamos. I am linking to excerpts. The book is still available for purchase. 
Advices to a Young mathematician, a collection of advice and anecdotes by M. Atiyah, B. Bollobas, A. Connes, D. McDuff and P. Sarnak.
A. Borel, Art and science (Math. Intelligencer vol.5 1983, translation from German). A text for a general audience about the relationship between art and mathematics.
R. P. Langlands Is there beauty in mathematical theories?, this text is actually about number theory, old and new.
T. Gowers The two cultures of mathematics, another take on the dichotomy between problem solving and theory building.
A. Connes A view of mathematics, a thorough exposition of A. Connes'philosophical stance about space and physics. Targeted at a scientific audience.
D. Mumford, the dawning of the age of stochasticity, from algebraic geometry to statistics.
Y. Manin, Interrelations between Mathematics and Physics, on the divergence between mathematics and physics in the XXe century.
M. Gromov, ergobrain, one of the most surprising inquiry about life and mathematics.
I end that list with a text from a french mathematician about the future of mathematics: Poincare, l'avenir des mathematiques.
A: Indiscrete Thoughts by Gian-Carlo Rota and Discrete Thoughts by Kac, Rota, and Schwartz.
A: Proofs and Refutations: The Logic of Mathematical Discovery by Imre Lakatos: The sequence of steps through which mathematical ideas can be made to grow in an  informal setting is explained through Socratic dialogues between a teacher and students. A beautiful read.
A: The Mathematical Experience by Philip J. Davis and Reuben Hersh is a wonderful collection of essays on mathematics and on the experiences and culture of mathematicians. Written back in the 1980's, it has extremely insightful discussions of many of the same topics that nowadays are discussed on MO. For example, the essay "The Ideal Mathematician," which describes a hypothetical "ideal" mathematician working on the made-up area of "non-Riemannian hypersquares" is absolutely hilarious. Highly recommended!
A: Love and Math: The Heart of Hidden Reality by Edward Frenkel is, in my opinion, a lot better than Lockhart's lament.  
A: Since you mentioned A Mathematician's Apology: Michael Harris' Mathematics Without Apology.
Here's an excerpt explaining the title:

These attempts at justifications are the 'apologies' of the title.  They usually take one of three forms.  Pure research in mathematics as in other fields is good because it often leads to useful consequences (Steven Shapin calls this the Golden Goose argument); it is true because it offers a privileged access to certain truths; it is beautiful, an art form.  To claim that these virtues are present in mathematics is not wrong, but it sheds little light on what is distinctively mathematical and even less about pure mathematicians' intentions.  Intentions lie at the core of this book.  I want to give the reader a sense of the mathematical life -- what it feels like to be a mathematician in a society of mathematicians where the first and second lives overlap.

A: I Want to be a Mathematician, by Paul Halmos.
A: Mathematics as Metaphor by Yuri Manin (both the title of the linked book which is a collection of essays, as well as the title of one particular essay in there).  At least some of the essays you can find online.
A: Perhaps a little broader in range/scope than the original question intended — but then again, perhaps not — the essays collected in
Mathématiques, mathematiciens et société.
Publications Mathématiques d'Orsay no. 86 74-16 (1974)
I was led to this when someone somewhere posted a link to Vergne's Témoignage d'une mathématicienne, which is one of the essays in this volume, and — I must confess — is the only one I've read, although the other ones do look interesting
A: In the Princeton Companion to Mathematics, there is a section entitled Advice to a Young Mathematician (pdf), containing essays by Atiyah, Bollobás, Connes, McDuff and Sarnak.
A: A Mathematician's Miscellany (reprinted, with additional material, as
Littlewood's Miscellany by CUP in 1986) is worthwhile reading.
Clifford Truesdell published a series of essays as An Idiot's Fugitive Essays on Science Methods, Criticism, Training, Circumstances (Springer, 1984), which sets out in a forthright manner the author's views on mathematics and science. 
A: A really nice article by Andrei Toom about mathematical education, especially in the US, got recently mentioned in a comment to this question.
