Question statement: Consider the space of probability measures with finite second moments $P_2(\mathbb{R}^d)$, which is equipped with the Wasserstein-2 distance $W_2$, and the square integrable function class $L_{2, \nu}(\mathbb{R}^D)$ with respect to a probability measure $\nu$. Let $S: P_2(\mathbb{R}^d) \rightarrow L_{2, \nu}(\mathbb{R}^D)$ be an operator such that $$(S\rho)(x) = \int \sigma(x; w) d\rho(w).$$ Here $\sigma$ is the "activation function", e.g., the sigmoid function. I am wondering if there are any sufficient conditions such that the operator $S$ is invertible and $S^{-1}$ is bounded.

My guess: For now I believe that $S$ is injective. My intuition comes from the universal function approximation theorem (UAT). Let $x = (x^0, \bar x)$ and $\sigma(x;w) = \sigma(w^T\bar x - x^0)$. Let $\rho_1,\rho_2 \in P_2(\mathbb{R}^d)$ such that $S\rho_1 = S\rho_2$. Assume that $$S\rho_1 - S\rho_2 = \int \sigma(\cdot; w) f(w) dw.$$ Then, by UAT, there exists a sequence of functions $f_n(w) = \sum_i b_i \sigma(w^T \bar x_i - x^0_i)$ such that $f_n \rightarrow f$ uniformly. Since $\int f_n f dw = 0$, it holds that $f = 0$ and $S\rho_1 - S\rho_2 = 0 \Rightarrow \rho_1 = \rho_2$. However, I am not sure if the above argument still holds true when $\rho$ does not have a density. Furthermore, I have no idea how to ensure that $S^{-1}$ is bounded.

  • $\begingroup$ Maybe I misunderstand something - but how can $S$ be surjective into $L_{2,\nu}(\mathbb{R}^D)$ when, actually, $S\rho \ge 0$ for each $\rho \in P_2(\mathbb{R}^d)$? $\endgroup$ – Jochen Glueck Feb 19 at 12:02

This is just a partial answer regarding $S$ being injective.

The generalisation of your argument is given by Hornik (Theorem 5 and the definition of discriminatory functions above Theorem 5)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.