Mathematics for machine learning I would like to know what mathematics topics are the most important to learn before actually studying the theory on neural networks.
I ask that because I will start to learn about neural networks and machine learning on my own to help in the analysis I am doing on my PhD about patterns of genome evolution. 
Thank you in advance.
 A: I think the answer depends on which structure you are looking to approximate, and, in what sense you want to approximate it.  Below, you'll find a few contemporary references to help out :)

Shallow Feedforward Networks and Deep Convolutional Networks
I would suggest some Harmonic/Fourier analysis, some constructive approximation theory, and their intersection (esp.: Besov Spaces).  This is because, many of the quantitative approximation theorems for shallow (1-hidden layer) feedforward networks are derived via such methods.  Relevant (contemporary) papers for such methods include:

*

*Approximation spaces of deep neural networks - Gribonval et al. 2021

*Approximation rates for neural networks with general activation functions - Siegel and Xu, 2021

*Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality - Taiji Suzuki
This particular point carries over equally to (deep) convolutional networks:

*Universality of deep convolutional neural networks - Ding-Xuan Zhou 2020

Deep Feedforward Networks and Optimal Rates
Otherwise, for deep feed-forward networks some of the more insightful approximation-theoretic results rely on Vapnik-Chervonekis Theory.  These are then typically used to derive "optimal approximation rates"; see especially these papers:

*

*Optimal approximation of continuous functions by very deep ReLU networks D. Yarotsky 2018

*The phase diagram of approximation rates for deep neural networks - Dmitry Yarotsky, Anton Zhevnerchuk - 2020

Non-Euclidean Input/Output Spaces and Topological Embeddings
These results typically rely on results of a more topological flavor.  I would Von Mill's book and of course basic general topology textbooks like Munkres' classic.  The only  universal approximation theorems I know of in this context are:

*

*Non-Euclidean Universal Approximation, 2020

*NEU: A Meta-Algorithm for Universal UAP-Invariant Feature Representation, 2021

Recurrent Structures and Reservoir Computers
If you're looking for something a bit more "dynamic" in nature, then I would recommend brushing up on your functional analysis, measure theory, and sequences in Banach spaces.  The first of these papers makes extensive use of ideas surrounding Rademacher Complexity and there are deep connections to the theory of dynamical systems.

*

*Risk Bounds for Reservoir Computing - Gonon, Grigoryeva, Ortega - 2020

*Differentiable reservoir computing - Grigoryeva, Ortega - 2019
I mention here also the developing connections between learning dynamics and rough path theory.  See:

*

*Deep signature transforms - Kidger, Bonnier, Perez Arribas, Salvi, Lyons - 2019

*Discrete-Time Signatures and Randomness in Reservoir Computing - Cuchiero, Gonon, Grigoryeva, Ortega, Teichmann - 2021

Qualitative Approximation by Shallow Feedforward Networks "Classical Style"
Let me mention that, classical (qualitative) universal approximation results are based on the Stone-Weierstraß theorem from approximation theory.  Some results rely on the theory of LF-Spaces which are a class of Locally-Convex spaces with a particularly "category-theoretic$\cap$functional-analytic flavor".  For modern formulations of the result in rather general contexts, see:

*

*Stone–Weierstraß and extension theorems in the nonlocally
convex case - Timofte, Timofte, Khan - 2018

*Stone-weierstraß theorems for group-valued functions - Galindo, Sanchis, 2004
The last of these references needs only a bit of background in topological groups.

Memory Capacity/ Interpolation Capabilities
These results have a variety of backgrounds.  The latter of these results draws from the Chow-Rashevskii Theorem and control theory.

*

*Small ReLu networks are powerful memorizers:  A tight analysis ofmemorization capacity - Yun, Sra, Jadbabaie - 2019

*Memory Capacity of Neural Networks with Threshold and Rectified Linear Unit Activations -  Vershynin - 2020

*Deep Neural Networks, Generic Universal Interpolation, and Controlled ODEs - Cuchiero, Larsson, Teichmann - 2020

Impossibility Theorems
Let me briefly round off this post with the following interesting results.  The pre-requisits for these papers are a typical background; nonetheless, their results are fascinating.

*

*Minimum Width for Universal Approximation - Park, Yun, Lee, Shin - 2021

*Deep, Skinny neural networks are not universal approximators - Johnson - 2018
A: I believe Ian Goodfellow and Yoshua Bengio's Deep Learning book covers the basics and also how you would use it for research. The chapters are also available online for free. 
A: Take a look at the web page for Michael Steele's course Probability Inequalities and Machine Learning, and the various texts linked to there.
A: As a deep learning practitioner with mathematical background I was yearning to have some satisfying mathematical framework of what I do in my every day job. In my opinion, very well fitted mathematical foundations of deep learning principles are captured simply by Empirical risk minimization (ERM) concept. I encourage everyone to read just Chapter 1 from The Nature of Statistical Learning Theory of V. Vapnik. In my opinion it is an eye opener.
A: For basic neural networks (i.e. if you just need to build and train one), I think basic calculus is sufficient, maybe things like gradient descent and more advanced optimization algorithms. For more advanced topics in NNs (convergence analysis, links between NNs and SVMs, etc.), somewhat more advanced calculus may be needed.
For machine learning, mostly you need to know probability/statistics, things like Bayes theorem, etc.
Since you are a biologist, I don't know whether you studied linear algebra. Some basic ideas from there are definitely extremely useful. Specifically, linear transformations, diagonalization, SVD (that's related to PCA, which is a pretty basic method for dimensionality reduction).
The book by Duda/Hart/Stork has several appendices which describe the basic math needed to understand the rest of the book.
