Please have a look at Jason Morton's work on Mathematics of Deep Learning. It is quite mathematically rigorous if that's what is you are asking for.  

Link: http://www.jasonmorton.com/publications.html ([Wayback Machine](https://web.archive.org/web/20200804164251/http://www.jasonmorton.com/publications.html))

Specifically, this [paper][1]

  [1]: https://benasque.org/2012network/talks_contr/106_morton.pdf