This is the first time I have posted a question on this site and it may not be suitable for this venue, which is primarily used for research questions in maths. If someone finds it unsuitable, it could be migrated to other sites like Maths StackExchange.

In a paper I am reading now, the author claims

... if some network computes a non-trivial function, and the product of its Schatten $p$-norm (for any $p<\infty$) is bounded, then there must be at least one parameter matrix, which is not far from being rank-1.

Mathematically, this means that if the output of network $N_{\mathbf{W}_{1}^d}(\mathbf{x})$ is not trivial, and $\prod_{i=1}^d\Vert \mathbf{W}_i\Vert_p\leq M$, then there exists $r\in \{1,2,\cdots,d\}$ such that $rank(\mathbf{W}_r)=1$.

However, the authors do not provide any proof of this claim. Furthermore, all their following theorems are motivated by this claim.

I am not sure if this is obvious, but could someone help me understand why this claim holds? Thank you in advance.


I just skimmed the paper, but it seems that this is later formalized into Theorem 3, a proof of which is in the appendix.

| cite | improve this answer | |
  • $\begingroup$ Thank you pointing this out! I just realized this. Actually I read the entire paper but maybe I am too involving into proof details and forgetting the general picture. $\endgroup$ – Mr.Robot Mar 6 '19 at 7:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.