# Best introduction to probability spaces, convergence, spectral analysis

I'm not sure if this stuff all falls under what most would just term "probability", but I'm researching applied macroeconomics and need to get a handle on the following concepts ASAP:

• probability spaces and sigma algebras
• Borel sets
• convergence
• stationarity/ergodicity
• martingales
• laws of large numbers
• spectral analysis

The text I'm using right now, Fabio Canova's Methods for Applied Macroeconomic Research, touches on all these things briefly in the space of about twenty-five pages, but it's pretty impenetrable. Does anyone know anything off-hand that presents these topics in a more practical, easier-to-digest way? This is a little bit better, but does anyone have any other suggestions?

-
As a mathematician, I really like the book by Durrett. But if you are not familiar with sigma algebras... maybe you should begin with a book on integration (for example the book by Rudin "Real and complex analysis"). Note that "ASAP" will probably require some time (a couple of month ?). Note also that you may get more answers on a forum devoted to teaching. –  camomille Apr 15 '11 at 15:38
Jacod and Protter's book is a reasonably friendly introduction, and doesn't require any background beyond multivariable calculus. As camomille said, you're looking at a couple of months of work to take all this stuff on board. –  Simon Lyons Apr 15 '11 at 16:00
Try "Measure, integral and probability", too. It doesn't cover all the topics you mentioned, but you should be able to work through it quite quickly, and it will establish a good base to build on. –  Simon Lyons Apr 16 '11 at 14:51

## 10 Answers

There is no royal road to probability. The closest is W. Feller's book, which has many (but not all) of the topics you mention, but I strongly advise reading (at least parts of) it first. Otherwise, you will go through life hopelessly confused.

-
After looking at most of the other books listed it seems like Feller's book is the easiest for the non-mathematician to pick up. Thanks for the recommendation. –  jefflovejapan Apr 16 '11 at 14:09

These don't have anything on spectral analysis, but cover the rest pretty well: Infinite Dimensional Analysis: A Hitchhiker's Guide by Aliprantis and Border, and Stochastic Limit Theory by Davidson. The first book was written for economic theorists, the second one for econometricians. So they specialize quite nicely to what you seem to be working on. However, they are both rather advanced.

A great introduction to measure theoretic probability is Probability with Martingales by Williams. It's quite chatty and fun, but does still require some mathematical sophistication. A book that I think is a bit dry but that proceeds in small and easy steps with all the details included is A Probability Path by Resnick.

-
+1 for mentioning Williams' Probability with martingales, aka the blue book. –  Did Apr 16 '11 at 5:56
I looked at Probability with Martingales, and I really enjoy the style, but it's a little out of my reach. I'm going to check out Hitchhiker's Guide on Monday. Davidson and Resnick aren't available at my library, but I put requests in for them. Thanks for the advice. –  jefflovejapan Apr 16 '11 at 14:11
Hitchhiker's guide is considerably more technical than Williams. It's very complete, but there's a danger of information overload there. –  Simon Lyons Apr 16 '11 at 14:49
The first two books are both great pure math texts, and its odd that they both come out of economics. –  arsmath Jun 30 '11 at 12:39

Many of the really good introduction-type books have already been mentioned. As a current grad student I encounter many of them on an almost daily basis and would suggest the following:

• Billingsley - "Probability and measure", although I would skip the first part about the dyadic intervals.

• Durrett - "Probability: Theory and examples". I used the 3rd version when I was taught from this book and then it did not have that much measure theory in the, sense that it was confined to the appendix. As I understand it this is not the case for the 4th edition and I really love the way Durrett presents the material so this is a good starting point.

• Shiryayev - "Probability". Great book from one of the current masters. It starts with an intuitive discussion about probability theory and then moves on to develop the mathematical theory needed (sample spaces and so on) in order to "do real probability". I haven't read the entire thing so that's why I can't put this as my #1 choice, I seems as if it has the potential though.

• Williams - "Probability with martingales". Very nicely written account of measure theoretic probability. It is quite concise though and depending on your mathematical maturity it could perhaps be a bit difficult to follow completely. One possibility is to keep the author's "Weighing the odds" at your side to get the elementary theory as a complement to the more advanced book.

• Feller - There is a reason why it's considered a classic.

Another, perhaps somewhat odd and unconventional, choice might also be Tomas Björk - "Arbitrage theory in continuous time". Now it's not an introduction to probability but depending on what type of economics you are interested this could be a nice reference book. It has an appendix devoted to measure theory and probability and Prof. Björk being one of the best teachers I've had in terms of providing you with an intuitive feeling for the subject at hand, I'm sure that such sections would work very well for someone who is just trying to get a basic understanding of the subject.

-

I think Probability and Measure by Patrick Billingsley is what you're looking for. It contains all of the topics you're asking about (except maybe spectral theory), and I feel it does a good job introducing the world of probability to the reader without assuming you already understand it, which Durrett's book tends to do.

-
It looks good, but the starting point's too advanced for me. Thanks for the advice. –  jefflovejapan Apr 16 '11 at 14:19

Those are probability topics, so I would go with what graduate students in probability are trained with. These past few years it seems that the standard is Rick Durrett's "Probability: Theory and Examples" (4th edition, Cambridge U. Press, 2010). MOst of what you want are in the first 5 chapters. Also a version of the book is provided in PDF format by the author, http://www.math.duke.edu/~rtd/PTE/PTE4_Jan2010.pdf. You could also get Shiryaev's "Probability" from Springer-Verlag or Stroock's "Probability Theory, an analytic view.

Hope that helps.

Cheers, Tipan

-

I agree with tipanverella that you're probably looking for textbooks designed for first-year graduate students in this field. I think you'd be hard-pressed to find one book that covers all the topics listed, but there's nothing wrong with reading parts of several books to get you up to speed. For everything except ergodic theory and spectral analysis, I highly recommend Probability Theory and Elements of Measure Theory by Heinz Bauer. This book has a fantastic first chapter and really develops $\sigma$-algebras from scratch (most books I've seen just throw the definitions at you). Throughout the development of measure theory, connections are made to probability and part II of the book is all probability theory. There's a whole chapter on the Law of Large Numbers and another on Martingales.

By the way, I would NOT read Rudin for this. Rudin's great if you already know the material and want a reference with the shortest, most elegant proofs possible (with many details skipped), but for actually learning the material and for connecting it to probability theory his book is not the greatest. Williams' Probability with Martingales really emphasizes exercises, so if you want examples with all the details worked out they will not be as prevalent. But the examples and exercises he gives are really great if you work through them yourself. I don't know much about the other books mentioned.

As for Ergodic Theory, I really like An Outline to Ergodic Theory by Steven Kalikow and Randall McCutcheon. If you click that link you can check out the table of contents and the introduction, which includes great examples to get you started as well as statements of all the big theorems you will need. I know this wasn't part of your question, but you may also want to add entropy to your list of things to learn. This book would help with that as well. Another thing you may find yourself needing is the Borel-Cantelli Lemma (found in Bauer's book).

Good Luck!

-

Given your starting point, specific interests, and desire to get to the bottom line fast, I recommend starting with Probability with Martingales by David Williams, which is yet another first-year graduate text, but more concise than the others.

-

OK ... "probability spaces" or "probability theory"?? Most of the other responses tell you texts for probability theory, where probability spaces are mentioned only in passing if at all. This would include the last 4 of your topics. On the other hand, "probability spaces" would be a branch of "measure and integration" theory.

-
That's an important distinction, but it certainly looks from the list of topics like the OP is interested in learning about probability theory, and most books on probability theory do devote a chapter to probability spaces before those recede to the background. –  Mark Meckes Apr 17 '11 at 23:54

The book "Probability and Stochastics" by Professor Erhan Cinlar is among the best introductory (actually it touches deeper things too) book on probability theory, assuming a little background on integration in general measure spaces. It starts with basic things (probability spaces, convergence, conditioning), and ends with some deeper waters like Levy-Ito for Levy processes, Hunt processes. Professor Cinlar is an expert in the theory of Markov processes (e.g. he co-authored papers that characterize Markov semimartingales), among many other things.

The book uses very precise language to describe things, in a very streamlined way. The author guides the reader through more difficult materials very easily, without making the reader feel the difficulty. In other words, pedagogically (in addition to the broad spectrum of things and the depth of the treatment) this book is among the best IMHO. I would recommend this book to anyone with an interest in probability theory and stochastic processes.

In terms of coverage of the OP's list, the book covers at least the following:

probability spaces and sigma algebras, Borel sets, convergence, martingales, laws of large numbers

-
Any thoughts on how Cinlar compares to Klenke, "Probability Theory A Comprehensive Course"? They both appear to be excellent recent alternatives to the likes of Billingsley and Chung. Also what would be good or required prep for Cinlar? Capinski and Kopp (seems to have a lot of overlap with Cinlar and Klenke)? Royden and Fitzpatrick (certain chapters)? –  user16097 Jun 30 '11 at 3:35

I just started up a blog and have put a link to my own alternative introduction to advanced probability theory. It could be useful as an introduction to Probability With Martingales, and might help you see the route ahead, as it were.

Here is the link:

http://fermatslastspreadsheet.wordpress.com/2011/11/30/introduction-to-probability-theory/

-