Question statement: Consider the space of probability measures with finite second moments $P_2(\mathbb{R}^d)$, which is equipped with the Wasserstein-2 distance $W_2$, and the square integrable function class $L_{2, \nu}(\mathbb{R}^D)$ with respect to a probability measure $\nu$. Let $S: P_2(\mathbb{R}^d) \rightarrow L_{2, \nu}(\mathbb{R}^D)$ be an operator such that $$(S\rho)(x) = \int \sigma(x; w) d\rho(w).$$ Here $\sigma$ is the "activation function", e.g., the sigmoid function. I am wondering if there are any sufficient conditions such that the operator $S$ is invertible and $S^{-1}$ is bounded.

My guess: For now I believe that $S$ is injective. My intuition comes from the universal function approximation theorem (UAT). Let $x = (x^0, \bar x)$ and $\sigma(x;w) = \sigma(w^T\bar x - x^0)$. Let $\rho_1,\rho_2 \in P_2(\mathbb{R}^d)$ such that $S\rho_1 = S\rho_2$. Assume that $$S\rho_1 - S\rho_2 = \int \sigma(\cdot; w) f(w) dw.$$ Then, by UAT, there exists a sequence of functions $f_n(w) = \sum_i b_i \sigma(w^T \bar x_i - x^0_i)$ such that $f_n \rightarrow f$ uniformly. Since $\int f_n f dw = 0$, it holds that $f = 0$ and $S\rho_1 - S\rho_2 = 0 \Rightarrow \rho_1 = \rho_2$. However, I am not sure if the above argument still holds true when $\rho$ does not have a density. Furthermore, I have no idea how to ensure that $S^{-1}$ is bounded.