Skip to main content
1 of 3
usul
  • 4.5k
  • 27
  • 30

Just saw this question. As KEW says in the comments, questions like these are studied in theoretical CS as it intersects with stats.

It depends on the measure used to define "good". One natural choice is the $\ell_1$ metric, so if the original distribution is the vector $A$, and your samples give the empirical distribution $\hat{A}$, then you want

$$ \|A - \hat{A}\|_1 = \sum_i |A_i - \hat{A}_i| \leq \epsilon .$$

In this case, it is "common knowledge" that $\Theta\left(\frac{n}{\epsilon^2}\right)$ samples are necessary and sufficient to ensure that, with probability $1-\delta$ for fixed $\delta$, $\|A - \hat{A}\|_1 \leq \epsilon$. I really don't think there is an original reference for this statement.

Another measure is the $\ell_{\infty}$ metric, so you want

$$ \|A - \hat{A}\|_{\infty} = \max_i |A_i - \hat{A}_i| \leq \epsilon . $$

In this case, it is "common knowledge" that $\Theta\left(\frac{1}{\epsilon^2}\right)$ samples are necessary and sufficient to ensure that, with probability $1-\delta$ for fixed $\delta$, $\|A-\hat{A}\|_{\infty} \leq \epsilon$. This I can at least attribute to the DKW inequality. (The argument in Ben Barber's answer gives the intuition, but not sure if it gives the exact answer because you would seem to have to union-bound over the coordinates.)

usul
  • 4.5k
  • 27
  • 30