Skip to main content
aligned comparisons
Source Link
user44143
user44143

The best difference is $\frac{1}{2m}$, attained by the distribution with $m$ atoms, each with mass $\frac{1}{m}$, at the points where the cdf of the normal distribution takes the values $\frac{2i-1}{m}$ for $i$ from $1$ to $m$.

Proof that this is optimal: One atom must have mass at least $\frac{1}{m}$. Call this $x$. Then $$\frac{1}{m} \leq F_m(x+\epsilon) - F_m(x-\epsilon) \leq |F_m(x+\epsilon) -\Phi(x+\epsilon) | + | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + |\Phi(x-\epsilon) - F_m (x-\epsilon)| \leq d_{KS} (F_m, \Phi)+ | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + d_{KS}(F_m,\Phi)$$\begin{align} \frac{1}{m} & \leq F_m(x+\epsilon) - F_m(x-\epsilon) \\ & \leq |F_m(x+\epsilon) -\Phi(x+\epsilon) | + | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + |\Phi(x-\epsilon) - F_m (x-\epsilon)|\\ & \leq d_{KS} (F_m, \Phi)+ | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + d_{KS}(F_m,\Phi) \end{align}

and as $\epsilon$ goes to $0$, $| \Phi(x+\epsilon) -\Phi(x-\epsilon)|$ goes to $0$ because $\Phi$ is continuous, so

$$\frac{1}{m} \leq 2 d_{KS}(F_m,\Phi)$$ showing the desired optimality.

Of course, this has nothing to do with the Gaussian, and works for any continuous distribution.

The best difference is $\frac{1}{2m}$, attained by the distribution with $m$ atoms, each with mass $\frac{1}{m}$, at the points where the cdf of the normal distribution takes the values $\frac{2i-1}{m}$ for $i$ from $1$ to $m$.

Proof that this is optimal: One atom must have mass at least $\frac{1}{m}$. Call this $x$. Then $$\frac{1}{m} \leq F_m(x+\epsilon) - F_m(x-\epsilon) \leq |F_m(x+\epsilon) -\Phi(x+\epsilon) | + | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + |\Phi(x-\epsilon) - F_m (x-\epsilon)| \leq d_{KS} (F_m, \Phi)+ | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + d_{KS}(F_m,\Phi)$$

and as $\epsilon$ goes to $0$, $| \Phi(x+\epsilon) -\Phi(x-\epsilon)|$ goes to $0$ because $\Phi$ is continuous, so

$$\frac{1}{m} \leq 2 d_{KS}(F_m,\Phi)$$ showing the desired optimality.

Of course, this has nothing to do with the Gaussian, and works for any continuous distribution.

The best difference is $\frac{1}{2m}$, attained by the distribution with $m$ atoms, each with mass $\frac{1}{m}$, at the points where the cdf of the normal distribution takes the values $\frac{2i-1}{m}$ for $i$ from $1$ to $m$.

Proof that this is optimal: One atom must have mass at least $\frac{1}{m}$. Call this $x$. Then \begin{align} \frac{1}{m} & \leq F_m(x+\epsilon) - F_m(x-\epsilon) \\ & \leq |F_m(x+\epsilon) -\Phi(x+\epsilon) | + | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + |\Phi(x-\epsilon) - F_m (x-\epsilon)|\\ & \leq d_{KS} (F_m, \Phi)+ | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + d_{KS}(F_m,\Phi) \end{align}

and as $\epsilon$ goes to $0$, $| \Phi(x+\epsilon) -\Phi(x-\epsilon)|$ goes to $0$ because $\Phi$ is continuous, so

$$\frac{1}{m} \leq 2 d_{KS}(F_m,\Phi)$$ showing the desired optimality.

Of course, this has nothing to do with the Gaussian, and works for any continuous distribution.

Source Link
Will Sawin
  • 148.4k
  • 9
  • 324
  • 563

The best difference is $\frac{1}{2m}$, attained by the distribution with $m$ atoms, each with mass $\frac{1}{m}$, at the points where the cdf of the normal distribution takes the values $\frac{2i-1}{m}$ for $i$ from $1$ to $m$.

Proof that this is optimal: One atom must have mass at least $\frac{1}{m}$. Call this $x$. Then $$\frac{1}{m} \leq F_m(x+\epsilon) - F_m(x-\epsilon) \leq |F_m(x+\epsilon) -\Phi(x+\epsilon) | + | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + |\Phi(x-\epsilon) - F_m (x-\epsilon)| \leq d_{KS} (F_m, \Phi)+ | \Phi(x+\epsilon) -\Phi(x-\epsilon)| + d_{KS}(F_m,\Phi)$$

and as $\epsilon$ goes to $0$, $| \Phi(x+\epsilon) -\Phi(x-\epsilon)|$ goes to $0$ because $\Phi$ is continuous, so

$$\frac{1}{m} \leq 2 d_{KS}(F_m,\Phi)$$ showing the desired optimality.

Of course, this has nothing to do with the Gaussian, and works for any continuous distribution.