21
$\begingroup$

I always find the strong law of large numbers hard to motivate to students, especially non-mathematicians. The weak law (giving convergence in probability) is so much easier to prove; why is it worth so much trouble to upgrade the conclusion to almost sure convergence?

I think it comes down to not having a good sense of why, practically speaking, a.s. convergence is better than convergence i.p. Sure, I can prove that one implies the other and not conversely, but the counterexamples feel contrived. I understand the advantages of a.s. convergence on a technical level, but not on the level of everyday life.

So my question: how would you explain to, say, an engineer, the significance of having a.s. convergence as opposed to i.p.? Is there a "real-life" example of bad behavior that we're ruling out?

Improve this question
$\endgroup$
2
  • 4
    $\begingroup$ en.wikipedia.org/wiki/… $\endgroup$
    – Steve Huntsman
    Commented Mar 24, 2010 at 1:08
  • 3
    $\begingroup$ I think it is in Chung's undergraduate probability textbook, he juxtaposes two quotations. (I don't have it here now, so cannot check my recolloction.) Both are quoted from famous mathematicians, one saying the weak law is useful and the strong law useless, and the other one saying the opposite. $\endgroup$
    – Gerald Edgar
    Commented Mar 24, 2010 at 1:13

4 Answers 4

Reset to default
19
$\begingroup$

Here is a nice post of T. Tao on SLLN. In the comments section he is asked a very similar question to which he answers the following: (I hope it's ok to reproduce it here, since it is buried down in the comments)

Returning specifically to the question of finitary interpretations of the SLLN, these basically have to do with the situation in which one is simultaneously considering multiple averages $\overline{X}_n$ of a single series of empirical samples, as opposed to considering just a single such average (which is basically the situation covered by the WLLN). For instance, if one had some random intensity field of grayscale pixels, and wanted to compare the average intensities at 10 x 10 blocks, 100 x 100 blocks, and 1000 x 1000 blocks, then the SLLN suggests that these intensities would be likely to be simultaneously close to the average intensity. (The WLLN only suggests that each of these spatial averages are individually likely to be close to the average intensity, but does not preclude the possibility that when one considers multiple such spatial averages at once, that a few outlying spatial averages will deviate from the average intensity. In my example with only three different averages, there isn’t much difference here, as the union bound only loses a factor of three at most for the failure probability, but the SLLN begins to show its strength over the WLLN when one is considering a very large number of averages at once.)

Improve this answer
$\endgroup$
1
7
$\begingroup$

Suppose you are in the context of collecting data and estimating the mean.

Imagine a situation where SLLN does not hold. It means that with positive probability accumulating new data is useless.

Improve this answer
$\endgroup$
1
  • $\begingroup$ But this positive probability isn't fixed: assuming the WLLN holds, then it's getting smaller as you accumulate more data, right? An example with made up values: while there is a 1% chance that getting 100 more data points is useless, there is only a 0.5% chance that getting 200 more data points is useless. If that's the case, getting more data is still guaranteed to be "useful" (albeit in a different sense) even if the SLLN doesn't hold. $\endgroup$
    – TheAmazingKitchen
    Commented Oct 2 at 11:56
5
$\begingroup$

I think it is worth noting that even if real world systems are fundamentally finite (in which case the distinction between WLLN and SLLN gets a bit philosophical), history has shown that it is extremely useful to approximate the discrete with the continuous. Thus we consider limit theorems to approximate statistics of large samples, we consider continuous distributions to approximate complicated finite distributions, and we consider continuous stochastic processes in order to approximate finite ones (e.g. Donsker's invariance principle).

The examples of sequences that converge in probability but not a.s. might seem a bit contrived, but then again most engineers seem to allow such philosophical absurdities as "let $X_n$ be an infinite sequence of coin tosses". In this regard, maybe it is best to phrase the distinction between convergence a.s. and convergence in probability in terms that seem more qualitative and less analytic. For example, imagine you were presented a sequence of gambles, and you must take either all of them or none of them. There is a very significant distinction between knowing that your wealth converges a.s. to some deterministic value vs knowing that it converges in probability (to that same value). In the former case, you expect in almost all states of the world that if you play the game that your wealth eventually stabilizes. However, in the case of convergence in probability you could go bankrupt infinitely often. Yikes!

Improve this answer
$\endgroup$
3
$\begingroup$

You might use the three-series theorem to elaborate on a.s. convergence. This approach would also have the advantage of working towards the SLLN.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .