I don't know about mathematics, but in science it can very useful to read the classic papers which were produced about the time that the ideas which you are learning came into being. This is often much better at giving someone a feel for the subject and how to actually do the subject, as opposed to being fed information via a very modern, efficient, streamlined teaching approach, which is not necessarily that good for giving people a feel for the way that the subject is actually done in practice.

Also I will just quote this from the Wikipedia article on Vladimir Arnold:

''He liked to study the classics, most notably the works of Huygens, Newton and Poincaré, and many times he reported to have found in their works ideas that had not been explored yet.''