If you have time, I highly recommend this Coursera course.
The videos are available for free and are truly excellent. The teacher is Geoffrey Hinton, who is one of the main players in the area, and he does an excellent job of providing both clear definitions and useful intuition.
In general, I wouldn't expect to see perfect theorem-lemma-proof exposition of deep learning anywhere, simply because the math hasn't caught up to real-world practice. More typical is a clean analysis of an idealized system, which is then related to a real system by a heuristic argument. In other words, this is an area that could use attention from mathematicians!