# What the the probability distribution of a mean?

There is an unknown set of values of unknown size, from which a known subset of N values is drawn at random.

Based on the known random subset, what is the probability distribution of the mean of the unknown larger set? Obviously, this probability distribution will become narrower for larger values of N, converging towards a single value for the mean.

Taking it a step further, is there an algorithm for selecting numbers at random from this probability distribution?

-
Without more information in the setup (eg a prior distribution for the "unknown set of values") this seems far from a mathematically well-posed question. Can you give more background? If there really is no more information than this, this comes down to the question "how do you first choose a prior distribution", which (as I understand it) is one of the places that statistics has to say "there's no general purely mathematical answer; this comes down to our real-world knowledge of the system in question". –  Peter LeFanu Lumsdaine Sep 29 '10 at 14:50
You can't get distribution of the mean, but you could get bounds on the mean of the form P(|unknownmean-knownmean|<k) which holds with some probability depending on sample size and k –  Yaroslav Bulatov Sep 29 '10 at 18:11

Suppose, for simplicity, that $N=\infty$, so you are just taking values from a distribution. In order to answer your question, you should have some prior knowledge about this distribution (i.e. a probability distribution in the space of all distributions). I will give you two examples of this.

Example 1: Suppose, you are doing (many times) some experiment, which has 2 results: 0 or 1. You don't know the probability p of "1", any value $p$ from 0 to 1 is possible. Then you can formalize your knowledge as the following: $p$ is uniformly distributed on $[0,1]$. Suppose, that after a few experiments you have got the sequence w="1010110111", you can write the formula, for a posterior density $f(p)$ of $p$. In general, if you have $n$ zeros and $m$ ones then $$f(p)=\frac{(1-p)^np^m}{\int_0^1(1-p)^np^mdp}=\frac{1}{n+m+1}\begin{pmatrix}n\\\\n+m\end{pmatrix}(1-p)^np^m.$$ This formula is just a continuous version of Bayes' theorem

Because mean of the result of experiment is exactly $p$, the formula, written above, is exactly formula for the distribution, you are searching for. If w="1010110111", then n=3, m=7 and this distribution looks like this:

Dotted line is at p=0.7.

Example 2: Suppose, result of your experiment is a real number. Then based on your case, you, for example, can consider it to be distributed normally with expectation $a$ and standard deviation $\sigma$. You don't know $a$ and $\sigma$, but you can consider some distribution law for them. For example, you can consider them to be independent, $a$ distributed with density $f_a(a)=\frac{10/\pi}{100+a^2}$ and $\sigma$ --- with density $f_\sigma(\sigma)=exp(-\sigma)$. After some experiments you can calculate posterior density of $f_\sigma$ and $f_a$. Since $a$ is exactly mean of the result of an experiment, $f_a$ is the density you are searching for.

-

"Based on the known random subset, what is the probability distribution of the mean of the unknown larger set?" With this question you've entered Bayesian land.

Added: Use an uniformative prior. This way you can still talk about the 'probability distribution of the mean of the unknown larger set' without frequentists getting on your case because you are presuming to know too much about the prior distribution.

-
Ok, but that doesn't really answer my question :-/ –  sanity Sep 29 '10 at 14:48
I would have made it a comment except that I'm only allowed to comment in my own answer threads because I don't have enough mathoverflow badges or achievements or whatever. –  ohai Sep 29 '10 at 14:58

"What is he probability distribution of the mean of the unknown larger set?" - you don't have enough information to answer this.

"Obviously, this probability distribution will become narrower for larger values of N" - not true. However large N becomes, you will still know nothing about possible extreme outliers of the unknown larger set.

"is there an algorithm for selecting numbers at random from this probability distribution?" - there can't be an algorithm for selecting the numbers, you have, by definition, to get them at random.

-