Maybe I didn't read the replies well enough, but apparently no one mentions A. Khinchin's book Mathematical Foundations of Statistical Mechanics. I just started reading this book. It's definitely mathematical and specifically says that it's written for a mathematician. The notation is a bit old, but the book is very readable, and far, far better than any book of physics I've seen lately.

It's only an introduction, so it doesn't contain the work done in recent decades, but a good one I would say. If there is anything really wrong with it, I would attribute it to the translator.