Recently uploaded paper in Arxiv. This paper formalize concepts and proves them. And few others that you might check out:
1 J. Bruna and S. Mallat, “Invariant Scattering Convolution Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1872–1886, Aug. 2013.
- for Covnet
[2] A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. LeCun, “The Loss Surfaces of Multilayer Networks,” arXiv:1412.0233 [cs], Nov. 2014.
- Spin Glasses Theory
- for Deep MLP and Convnet
[3] P. Mehta and D. J. Schwab, “An exact mapping between the Variational Renormalization Group and Deep Learning,” arXiv:1410.3831 [cond-mat, stat], Oct. 2014.
- for Restricted Boltzmann Machine, Deep Belief Network and Deep Boltzmann Machine.