20
$\begingroup$

Could someone please recommend reading on Bayesian statistics presented from a pure mathematical point of view? That is, works that start assuming a good knowledge of measure theoretic probability. The usual reference works for statisticians gloss over the fine details, but for me this just leads to more confusion!

I already know of the following question

Statistics for mathematicians

and will follow up some of the suggestions there, but I am specifically looking for a treatment of the Bayesian approach, so if there are any recommendations that apply just to this I would appreciate hearing about them.

$\endgroup$
  • 1
    $\begingroup$ I would say that engineering books are more easy to read and to get main ideas, examples and motivations. May be like this one amazon.com/… $\endgroup$ – Alexander Chervov Jun 22 '12 at 15:12
  • $\begingroup$ Tom, did you end up finding a book to your liking? I too wish to understand the Bayesian framework from a rigorous, pure math standpoint, so I'd appreciate any suggestions or advice. $\endgroup$ – goblin May 2 '16 at 9:11
  • $\begingroup$ Hi @goblin, I'm afraid I can't recommend anything specific. $\endgroup$ – Tom Ellis May 5 '16 at 10:21
13
$\begingroup$

Many hold that Bayesian statistics "from a purely mathematical point of view" is entirely coextensive with probability (however it is that you want to define its boundaries as a mathematical discipline). Nonetheless, if I interpret your request as being for a mathematically sophisticated and rigorous exposition on why the Bayesian approach is a worthy one, three book spring to mind.

  1. Theory of Statistics by Mark Schervish
  2. Bayes Theory by John Hartigan
  3. The Bayesian Choice by Christian Robert

The first of these is a general graduate text in statistics, but the author gives uncommonly complete coverage of both Bayesian and frequentist methods.

The second is a smaller volume and, as I recall, is devoted to some of the more delicate issues surround finite versus countable additivity as relates to using probability distributions as priors in a Bayesian approach.

The final book is more general, but the style is more formal than the Bernardo and Smith book mentioned by PaPiro. (This is, in my experience, true of the style of French Bayesians :)

As I said, the distinctive elements of the Bayesian perspective are more philosophical than technical, but there are some technical areas that have received attention in the Bayesian community that may be of independent mathematical interest. One would be the role of so-called "improper" priors as mentioned above.

Another is the role of conditional distributions as a primitive rather than derived notion, leading to the idea of disintegration, as in this manuscript of Pollard.

Also, because of a keen interest in the application of Monte Carlo methods, Bayesian statisticians have to a lot of work on various aspects of computational methods for sampling from various distributions. Christian Robert is a prominent researcher in this area, and he has a blog. The current post happens to be about Bayesian foundations.

Finally, at the heart of a many arguments in favor of a Bayesian approach (early chapters in Bernardo and Smith and Robert are dedicated to it) are de Finetti type representation theorems, which sanction prior distributions via appeals to exchangeability. You can start with the wiki entry for de Finetti theorems and then look at the work of Persi Diaconis on the topic. In this vein see also Lauritzen's monograph, which (for me anyway) is the last word on the matter.

$\endgroup$
  • 4
    $\begingroup$ There is a recent paper on the arXiv that takes conditional distributions as the foundation of Baysian statistics very seriously: "A categorical foundation for Bayesian probability" by Culbertson and Sturtz, arxiv.org/abs/1205.1488 $\endgroup$ – Michael Greinecker Jun 21 '12 at 22:50
  • $\begingroup$ @Michael Greinecker Thanks for the reference, I'll definitely take a look. It isn't my area of research, but I have an abiding interest in keeping up to date on the current thinking about such things. $\endgroup$ – R Hahn Jun 21 '12 at 23:27
  • 1
    $\begingroup$ +1 for mentioning Schervish's book. It's the best, imho. (There are a fair number of minor typos, but I think that's excusable for a 700-page first edition.) $\endgroup$ – William DeMeo Jun 22 '12 at 5:17
3
$\begingroup$

Suggestions:

  1. W.M. Bolstad, Introduction to Bayesian Statistics 2nd. Ed., Wiley, 2007.
  2. J.M. Bernardo, A.F.M. Smith, Bayesian Theory, Wiley, 2000.
  3. J.K. Ghosh, M. Delampady, T. Samanta, An Introduction to Bayesian Analysis: Theory and Methods, Springer, 2006.

Freely avaliable:

  1. J.F. Kenney, Mathematics of Statistics Part One, Chapman & Hall Ltd. 1939.

  2. J.F. Kenney, Mathematics of Statistics Part Two, Chapman & Hall 1940.

$\endgroup$
3
$\begingroup$

In addition to former proposals, why not the famous two-volume work "Theory of Probability" by Bruno de Finette (In the Wiley classics series)

Quotation: "Probability do not exist"

A more modern book, with quite another take than the above, is "Bayesian Data Analysis", Third Edition by A Gelman, J B Carlin, H S Stern, D B Dunson, A Vehtari, and D B Rubin (CRC press). This a modern book with emphasis on data analysis and computation. I would say that for today (2017) this is the best starting point.

$\endgroup$
3
$\begingroup$

There is a book of 2018 on the topic, that also presents some recent results of the author:

S. Watanabe, Mathematical Theory of Bayesian Statistics, Chapman and Hall/CRC, Apr 2018

$\endgroup$
  • 1
    $\begingroup$ Unlike the provided link, the Amazon page has a preview of the book: amazon.com/Mathematical-Bayesian-Statistics-Chapman-Monographs/…. The claims in the provided link are vague and bold, mentioning "new statistical laws". Section 9.4 on "phase transitions" is intriguing, but the definition given is very ambiguous. $\endgroup$ – R Hahn Jul 16 '18 at 18:26

protected by Community Jul 21 '17 at 9:08

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.