I worked with neural nets in the 80's and haven't kept up with the literature, so I don't have any links. However, I came away with the conclusion that training neural nets is essentially training a digital computer to behave like an analog computer. So, examining the mathematics pertaining to analog circuits might give some insight in this area.