Let me first pose a trivial question.

Given a Borel probability measure $\mu$ on the real line, is it possible to construct a purely atomic random measure $M$ whose mean is $\mu$?

The answer is obviously yes: take $M = \delta_{X}$, where $\delta_x$ is the Diract measure sitting at $x$ and $X$ is a random variable distributed according to $\mu$. In fact take $M$ to be the empirical measure $\frac{1}{n} \sum_{i=1}^n \delta_{X_i}$ where each $X_i \sim \mu$ suffices.

Now here is my question:

Given a Borel probability measure $\mu$ on the real line and $\delta > 0$, is it possible to construct a purely atomic random measure $M$ whose mean is $\mu$ and $d(M, \mu) < \delta$ almost surely, where $d$ is some metric on the space of probability measures (e.g. the Wasserstein distance, the Lévy–Prokhorov metric or the Kolmogorov distance, i.e., the sup-distance between distribution functions)?

Of course for an arbitrary distance this is not always possible. For example, if $d$ is the total variation distance and $\mu$ is atomless, then $d(M,\mu) = 2$ a.s. But for a weaker distance, will this be possible? The intuition is the following: consider the empirical measure $\frac{1}{n} \sum_{i=1}^n \delta_{X_i}$ where $X_i$ are iid generated from $\mu$. Then as $n\to\infty$, it will converge to the mean $\mu$ a.s. under those distances. Therefore within any $\delta$-ball centered at $\mu$, there are lots of atomic measures, i.e., $\mathbb{P} \{d(M,\mu) < \delta\}$ is very close to 1. But can we achieve exactly 1, i.e., can we construct an $M$ supported on those measures which are close to the desired mean?