136
$\begingroup$

I'm looking for an overview of statistics suitable for the mathematically mature reader: someone familiar with measure theoretic probability at say Billingsley level, but almost completely ignorant of statistics.

Most texts I've come across are either too basic, or are monographs focused on a specific area or technique.

Any suggestions?

Improve this question
$\endgroup$
4
  • 9
    $\begingroup$ There's a reason for that. The basics of statistics don't require measure theory; they're hampered by it. Anyway, Schervish's "Theory of Statistics" may be up your alley. $\endgroup$
    – Shiva Kaul
    Commented Jul 13, 2010 at 4:53
  • 11
    $\begingroup$ Answer from statistical community can be found stats.stackexchange.com/questions/414/… $\endgroup$
    – robin girard
    Commented Aug 2, 2010 at 16:47
  • 4
    $\begingroup$ I have answered before a similar question here: stats.stackexchange.com/questions/63760/… $\endgroup$
    – kjetil b halvorsen
    Commented Oct 9, 2013 at 16:42
  • $\begingroup$ I don't know what is "Billingsley level" (although I have background on measure theory and basic probabilities following a measure theory course). $\endgroup$
    – YCor
    Commented Dec 7, 2022 at 15:55

15 Answers 15

Reset to default
79
$\begingroup$

This won't directly answer the question, but here are some things a mathematician who wants to learn about statistics should learn:

  • When is a random variable a statistic and when is it not? (A statistic is an observable random variable. For example $X - E(X)$ is not a statistic if the "population average" $E(X)$ is not observable.
  • Fisher's concept of sufficiency. Examples, characterizations, theorems. In particular, the Rao--Blackwell theorem and examples of its use. That's way cool.
  • So is the concept of completeness and the Lehmann--Scheffe theorem.
  • If you think that linear regression is called linear because you're fitting a line, then you are naive. If you're fitting, e.g., a parabola by finding least-squares estimators of three parameters, then you're doing linear regression. There is also such a thing as non-linear regression.
  • Learn the Gauss--Markov theorem on Best Linear Unbiased Estimators (BLUEs).
  • Look at my recent answer to a question on prediction intervals. Why do you need the (finite-dimensional version of) the spectral theorem to understand linear regression? (Look at the aforementioned answer and consider this question an exercise.)
  • As long as we're on linear regression (the topic of the three bullets immediately above this one), look at the Wikipedia article titled "errors and residuals in statistics" (written mostly by me). Learn the difference between an error and a residual. Maybe look at "Studentized residual" as an afterthought.
  • ....and then at "lack-of-fit sum of squares".
  • If you think linear regression is child's play rather than something to which the most brilliant person could devote a long career in research, grow up.
  • Learn the difference between frequentism and Bayesianism. In fact, look at the rant I posted on nLab about this. (The essence of Bayesianism is that probabilities are taken to be epistemic. Bayesianism is not more subjective than frequentism; rather Bayesians and frequentists put their subjectivity in different places. (A really glaring example is the 5% critical value legendarily used in medical journals. Why 5%? Because that's a subjective economic choice.))
  • Learn design of experiments. Learn why Latin squares and a myriad of other combinatorial designs are used.
  • OK, maybe a small and incomplete but nonetheless direct answer to the original question: perhaps Hocking's book on linear models.
  • Learn to use the word "sample" correctly. If you ask the next 100 people you meet whether they intend to vote "yes" or "no", that's not 100 samples; that's one sample.
  • Another thing that will give you some idea of the distinct flavor of the subject, and how it differs from probability theory and some other fields, is books on sampling.
  • Learn about the Wishart distribution.
  • And the multivariate normal distribution.
  • Exercise: How do you prove that every non-negative-definite matrix is the variance of some random vector?
  • Learn why the Behrens--Fisher problem cannot be regarded as a math problem. It belongs up there with Hilbert's problems as one of the great challenges, but it's not mathematics for this reason: One can model it as a math problem in any of a variety of different non-equivalent ways. One can solve those math problems. But which one is the "right" model? That's essentially a philosophical question. And that question, not the math problems, is the Behrens--Fisher problem. (The Behrens--Fisher problem is this: how do you draw inferences about the difference between the means of two normally distributed populations which may have different variances? "Inferences" can mean point-estimates or interval estimates or perhaps other things.)

This is just a sampling of the first things that come immediately to mind. It leans toward showing you what the subject tastes like rather than what it's important to know to do theoretical or applied research.

Statistics is an immensely broader field than mathematical probability theory.

Improve this answer
$\endgroup$
7
  • 4
    $\begingroup$ A lot of stuff to chew over there. perhaps you could insert one or two links (such as for that nLab entry you mention)? $\endgroup$
    – Yemon Choi
    Commented Jul 13, 2010 at 21:14
  • 6
    $\begingroup$ What does "Statistics is an immensely broader field than mathematical probablity theory" means to you as a statistician :) ? is it a troll you try to conclude with ? Anyway, thanks for all those references. My point of view (as a statistician): it is a personnal/subjective list. It is not in line with the question (becasue it is not an overview) but I like the fact that it is a fresh and original answer. Maybe it it not the good place for this answer and you should create another question such as "what are the particular subject/principles of statistic a mathematician should learn first?" $\endgroup$
    – robin girard
    Commented Jul 15, 2010 at 6:35
  • 2
    $\begingroup$ Well, I find the assessment that it's personal and subjective to be in accord with my own remarks near the end. "Broader" means more has been done and the diversity of topics (even if one sticks fairly close to the parts of it that are just mathematics) is greater. Maybe no one's written a book that is to statistics what Boolos & Jeffrey's Computability and Logic is to mathematical logic. That book is addressed to mathematicians who want to find out what mathematical logic is about, and it goes more from breadth than for depth, although almost all of it is strictly rigorous. $\endgroup$
    – Michael Hardy
    Commented Jul 15, 2010 at 20:42
  • 23
    $\begingroup$ I think that "Statistics is an immensely broader field than mathematical probablity theory" is as reasonable a statement as "Mathematics is an immensely broader field than mathematical probablity theory." And as useful. $\endgroup$
    – Mark Meckes
    Commented Jul 21, 2010 at 14:05
  • 14
    $\begingroup$ I like this answer, just wish it weren't so polemical. $\endgroup$
    – Jim Conant
    Commented Dec 12, 2018 at 20:45
23
$\begingroup$

Wasserman's All Of Statistics seems concise and useful.

Improve this answer
$\endgroup$
3
  • 4
    $\begingroup$ Hi. I am considering using this book, Is it rigorous ? Thank you $\endgroup$
    – Amr
    Commented Dec 18, 2013 at 12:05
  • 13
    $\begingroup$ Wasserman's book is not rigorous. Not by a long shot. $\endgroup$
    – Michael
    Commented Jul 1, 2014 at 15:46
  • $\begingroup$ I can recommend this book. $\endgroup$
    – mathoverflowUser
    Commented Oct 27, 2022 at 13:56
20
$\begingroup$

David Williams "Weighing the Odds" is a book about probability and statistics by a distinguished probability theorist. He has a great sense of humour and the book is a lot of fun. The exercises can be demanding, but they are also interesting. (I have latex solutions to many of them if you want to ask me after first trying an exercise.) Professional statisticians think this book is more maths than stats, but it does contain a lot of stats. It's an introduction that will hold your attention if you put effort into the exercises. It's aimed at advanced undergraduates, but in my opinion is also very suitable for research mathematicians (though I have my doubts as to its suitability for a professional probability theorist).

After reading and enjoying this book, a mathematician will find it much easier to get to grips with a good book on stats. But there's no substitute for trying to handle large amounts of real data, when nothing works out as it "ought" to. Learning stats without handling data is like learning maths without working out anything for yourself---it's too easy to kid yourself that you understand.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I agree William's book is a treasure. It contains many interesting points which are not easy to find elsewhere. $\endgroup$
    – Stéphane Laurent
    Commented Jun 23, 2012 at 9:58
  • $\begingroup$ Just read the preview on Amazon and I'm going to the library to grab it now! $\endgroup$
    – Taiki
    Commented Mar 31, 2014 at 14:01
  • 1
    $\begingroup$ I read his book "Probability with Martingales" and his writing style is indeed very enjoyable. $\endgroup$
    – Luc M
    Commented Nov 27, 2019 at 16:28
10
$\begingroup$

Probability Theory: the Logic of Science by physicist E. T. Jaynes is a must.

Improve this answer
$\endgroup$
1
  • 9
    $\begingroup$ A "alternative" view. $\endgroup$
    – Michael Hardy
    Commented Dec 26, 2010 at 1:59
6
$\begingroup$

I'm currently working my way through Cramér's Mathematical methods of statistics. It starts out with a half-book primer on all measure theory, Lebesgue integration et.c. you might possibly need for anything, and then goes through first probability and then statistics with this backdrop.

Skipping the introductory analysis it might be what you're looking for.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Cramer's book is still a classic and well worth a look. $\endgroup$
    – The Mathemagician
    Commented Jul 13, 2010 at 16:45
4
$\begingroup$

I liked Mathematical Statistics by Jun Shao for the classical theory. There is also a companion 'Problems & Solutions' book.

I second Wasserman's 'All of Statistics' for a quick overview.

Improve this answer
$\endgroup$
4
$\begingroup$

I like "Statistics" by Freedman, Pisani, Purves, and Adhikari

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ That book is also suitable for the mathematically immature reader. It's amazing how much they do with virtually no math. (Of course, the book also omits a lot of fairly basic stuff. I've heard it criticized on the grounds that it assumes everything is normally distributed.) $\endgroup$
    – Michael Hardy
    Commented Dec 11, 2010 at 17:53
  • 1
    $\begingroup$ Read at stats.stackexchange.com/a/1666/11887 praise of that book by Cross Validated top contributor "whuber", just now passing 300k rep! $\endgroup$
    – kjetil b halvorsen
    Commented Dec 7, 2022 at 14:35
4
$\begingroup$

There is a book I love: "Statistical inference" by S.D. Silvey. Short, elementary, but perfectly clear and suited for the mathematical mind. Impossible not to like statistics after that.

Improve this answer
$\endgroup$
4
$\begingroup$

It's old fashioned, but it's written by a master and most of it may just be what you're looking for: Mathematical Statistics by S.S.Wilks. It's long out of print and hard to find, but worth hunting down.

Improve this answer
$\endgroup$
3
$\begingroup$

Mark J. Schervish "Theory of Statistics" perhaps is the one closest to your needs since it

  1. is graduate-student level and proves everything rigorously from probability measure perspective
  2. has a rather broader view comparing with other monographs
  3. gets 5 stars on Amazon

But as the some ppl pointed out, statistics are too broad and have been under a intensive development till now. This book missed some modern results as expected.

Improve this answer
$\endgroup$
3
$\begingroup$

Pestman, Wiebe R., Mathematical statistics., de Gruyter Textbook. Berlin: de Gruyter (ISBN 978-3-11-020852-8/hbk; 978-3-11-020853-5/ebook). ix, 597 p. (2009). ZBL1251.62001.

It can't be said to be advanced in that it is intended for undergraduates, but it is written in the style of a book for the mathematically mature. I.E. Definition, Theorem, Proof, Example. It even begins with a brief introduction to measure theory. That's a big improvement over most!

Improve this answer
$\endgroup$
3
$\begingroup$

The Book of Van der Vaart is exactly what you are searching for!

Asymptotic statistics

Improve this answer
$\endgroup$
2
$\begingroup$

Kiefer's "Introduction to Statistical Inference" is a particularly nice book. It crams an incredibly amount of perspective into a very easy to read package. You'll want to supplement it with volumes like Lehmann's "Point Estimation" and "Testing Statistical Hypotheses."

One thing I particularly like is that he starts at the bottom with the decision theory structure, and imposing other criteria to be able to choose from the plethora of admissible procedures before diving into things like linear estimators and hypothesis tests.

Improve this answer
$\endgroup$
2
$\begingroup$

Mathematical Statistics, by Jun Shao, the most mathematical statistics book that I know. This book is mainly focused on classical statistics. If you want to proceed further into classical statistics, E.L. Lehmann's two books Theory of Point Estimation and Testing Statistical Hypothesis are bibles of classical statistics, worth your reading.

High-Dimensional Statistics: A Non-Asymptotic Viewpoint, by A.M. Wainwright, another popular book, also quite mathematical, mainly focused on high dimensional statistics.

Enjoy!

Improve this answer
$\endgroup$
2
$\begingroup$

For a very mathematical version of statistics, my favorite is online lecture notes from two MIT courses. The instructor is named Panchenko and the course is called 'Statistics for Applications'. There are course notes that read like a book for the course in 2003 and 2006. I have enjoyed browsing through both of them. Here is a link for the 2006 course: https://ocw.mit.edu/courses/18-443-statistics-for-applications-fall-2006/

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .