There is an unknown set of values of unknown size, from which a known subset of N values is drawn at random.
Based on the known random subset, what is the probability distribution of the mean of the unknown larger set? Obviously, this probability distribution will become narrower for larger values of N, converging towards a single value for the mean.
Taking it a step further, is there an algorithm for selecting numbers at random from this probability distribution?
Suppose, for simplicity, that $N=\infty$, so you are just taking values from a distribution. In order to answer your question, you should have some prior knowledge about this distribution (i.e. a probability distribution in the space of all distributions). I will give you two examples of this.
Example 1: Suppose, you are doing (many times) some experiment, which has 2 results: 0 or 1. You don't know the probability p of "1", any value $p$ from 0 to 1 is possible. Then you can formalize your knowledge as the following: $p$ is uniformly distributed on $[0,1]$. Suppose, that after a few experiments you have got the sequence w="1010110111", you can write the formula, for a posterior density $f(p)$ of $p$. In general, if you have $n$ zeros and $m$ ones then $$f(p)=\frac{(1-p)^np^m}{\int_0^1(1-p)^np^mdp}=\frac{1}{n+m+1}\begin{pmatrix}n\\\\n+m\end{pmatrix}(1-p)^np^m.$$ This formula is just a continuous version of Bayes' theorem
Because mean of the result of experiment is exactly $p$, the formula, written above, is exactly formula for the distribution, you are searching for. If w="1010110111", then n=3, m=7 and this distribution looks like this:
Dotted line is at p=0.7.
Example 2: Suppose, result of your experiment is a real number. Then based on your case, you, for example, can consider it to be distributed normally with expectation $a$ and standard deviation $\sigma$. You don't know $a$ and $\sigma$, but you can consider some distribution law for them. For example, you can consider them to be independent, $a$ distributed with density $f_a(a)=\frac{10/\pi}{100+a^2}$ and $\sigma$ --- with density $f_\sigma(\sigma)=exp(-\sigma)$. After some experiments you can calculate posterior density of $f_\sigma$ and $f_a$. Since $a$ is exactly mean of the result of an experiment, $f_a$ is the density you are searching for.
"Based on the known random subset, what is the probability distribution of the mean of the unknown larger set?" With this question you've entered Bayesian land.
Added: Use an uniformative prior. This way you can still talk about the 'probability distribution of the mean of the unknown larger set' without frequentists getting on your case because you are presuming to know too much about the prior distribution.
"What is he probability distribution of the mean of the unknown larger set?" - you don't have enough information to answer this.
"Obviously, this probability distribution will become narrower for larger values of N" - not true. However large N becomes, you will still know nothing about possible extreme outliers of the unknown larger set.
"is there an algorithm for selecting numbers at random from this probability distribution?" - there can't be an algorithm for selecting the numbers, you have, by definition, to get them at random.
