Return to Revisions

Output of the ReLU network is $$v = \sum_{ij} X_i A_{ij} w^{(1)}_{ij} \cdots w^{(L)}_{ij} $$ where $i$ is the input $j$ is the path and $A_{ij}$ is $1$ if the path is open and $0$ otherwise. Now if you take the expectation and use independence of the noise and also assume that the noise has mean $0$ we get that asymptotically when $m$ is large we have $\mathbb{E}(\lVert v - v^* \rVert_2) \leq 1/10^L$ since $A_{ij}$ and the product of parameters are asymptotically independent. Note that the the expectation of the output is a sum of the expectation of the original network and the expectation of the network that contains only the noise.