Here is a sufficient condition for non-injectivity under the assumption that $w=(\omega,\beta)$ and $\sigma(x;w)=\sigma(\omega\cdot x + \beta)$ i.e. an artificial neuron with activation $\sigma$, weights $\omega$, and bias $\beta$.  

**Claim:** If $\sigma:\mathbb R\to\mathbb R$ is positive-homogenous (i.e. $\sigma(\lambda x)=\lambda \sigma(x)$ for all $\lambda\geq 0$ and $x\in\mathbb R$) then $S\delta_w=S(\tfrac{1}{2}\delta_{2w}+\tfrac{1}{2}\delta_0)$ for all $w$ and so $S$ is not injective.

*Proof.* To see why, note that positive-homogeneity implies $\sigma(x;2w)=2\sigma(x;w)$ and $\sigma(x;0)=0$ and
$$ 
S[\delta_w](x) = \int \sigma(x;w')\,\mathrm d\delta_w(w') = \sigma(x;w)
$$
Thus it follows that
$$
 S\left[\tfrac{1}{2}\delta_{2w}+\tfrac{1}{2}\delta_0\right](x) = \tfrac{1}{2}S[\delta_{2w}](x) + \tfrac{1}{2}S[\delta_0](x) = \tfrac{1}{2}\sigma(x;2w) + \tfrac{1}{2}\sigma(x;0) = \sigma(x;w)
$$
Since $\delta_w\neq \tfrac{1}{2}\delta_{2w}+\tfrac{1}{2}\delta_0$ as measures (unless $w=0$) it follows that $S$ is not injective. $\blacksquare$

An example of such an activation function is the commonly used Rectified Linear Unit (ReLU) $\text{relu}(x)=\max(x,0)$. Thus merely appealing to the universal approximation theorem will not suffice in a search for an injectivity condition on $S$ as single-hidden layer neural networks with ReLU activations [are universal][1].


  [1]: https://www.sciencedirect.com/science/article/abs/pii/S0893608005801315