-1
$\begingroup$

if we are given a finite number N of points drawn from a probability distribution, expectation can be approximated as a finite sum over these points: E[f]=(1/N)(summation of f(x) over these N points).

comparing this to the actual calculation of E[f]=summation of p(x)f(x), won't the difference between the actual value and approximate value be a lot in cases where p(x) varies a lot?

Improve this question
$\endgroup$
2
  • 4
    $\begingroup$ Yes it might. The standard deviation of the sample mean will be large if the underlying distribution has too large standard deviation (if that's what you mean by "varies a lot"). But this is far from a research level question, so not really suitable here. Maybe math.stackexchange would be a better fit? $\endgroup$
    – George Lowther
    Commented Jan 20, 2011 at 0:24
  • $\begingroup$ Or stats.stackexchange.com. You are interested in something like the variance of the sample mean. $\endgroup$
    – Nate Eldredge
    Commented Jan 20, 2011 at 3:57

1 Answer 1

Reset to default
3
$\begingroup$

The Strong Law of Large Numbers guarantees almost sure convergence of the sample mean to the population mean. If your distribution has large variance then yes the convergence is slower. However, the probability of being away from the population mean is bounded by:

$P(|s_n-\mu|>\epsilon)<\frac{\sigma^2}{n\epsilon^2}$

Where $\mu$ and $\sigma$ are true mean and standard deviation and $s_n$ is the sample mean from $n$ points.

Improve this answer
$\endgroup$
4
  • $\begingroup$ It's worth commenting that all bets are off if the population has a distribution that doesn't have a finite variance. $\endgroup$
    – Brian Borchers
    Commented Jan 20, 2011 at 0:44
  • 1
    $\begingroup$ @Brian: the SLLN holds with no assumption on the variance. It requires only the assumption of finite mean. $\endgroup$
    – Qiaochu Yuan
    Commented Jan 20, 2011 at 0:49
  • $\begingroup$ Are there rate of convergence estimates when dealing with infinite variance? I would guess one would have to truncate the random variables... $\endgroup$
    – Alex R.
    Commented Jan 20, 2011 at 1:30
  • $\begingroup$ @Alex Yes, there are long books written on the subject (see, eg, Gnedenko/Kolmogorov, or Feller v II, especially the section on Stable Laws). $\endgroup$
    – Igor Rivin
    Commented Jan 20, 2011 at 3:26

Not the answer you're looking for? Browse other questions tagged .