I apologize for my ignorance, but hope that someone would provide some pointers to what I am sure is a reasonably well-developed body of theory.  Consider $C^\infty(U,V)$ where $U \subset R^k$ and $V \subset R^l$ - I am reasonably sure that this space can be given the structure of a an infinite dimensional manifold.  Suppose now that I single out some well-defined class of functions, each element of which is specified by a finite collection of parameters.  In particular, I am interested in feed-forward neural networks of fixed architecture, so that the weights and biases give a parametrization, and I would like to think of training as a curve in this class of functions.  Does such a collection have the structure of a non-compact finite-dimensional embedded submanifold, in some topology on $C^\infty(U,V)$?  What are some references that might cover such topics?  I have had standard graduate courses in differential and Riemannian geometry, but am woefully ignorant in functional analysis.  I would be grateful for any pointers to sources that might be relevant.