14
votes
Mathematics for machine learning
I think the answer depends on which structure you are looking to approximate, and, in what sense you want to approximate it. Below, you'll find a few contemporary references to help out :)
Shallow ...
- 5,405
14
votes
Structures of the space of neural networks
I would like to argue that the space of neural networks is a category with finite products, or more concretely a Lawvere theory. This expresses an important piece of structure, namely how neural ...
- 6,406
13
votes
Is there any paper which summarizes the mathematical foundation of deep learning?
Mathematics of Deep Learning (2017)
This tutorial will review recent work that aims to provide a
mathematical justification for several properties of deep networks,
such as global optimality, ...
- 188k
10
votes
Accepted
Structures of the space of neural networks
In information geometry, people study structures of Riemannian manifolds with dual affine connections on sets of neural networks. (The metric measures how close neural networks are in their input-...
- 17.4k
9
votes
Neural networks over gadgets other than $\mathbb{R}$
Studies of neural networks that are more general than $\mathbb{R}^n\mapsto\mathbb{R}^k$ include
Deep Complex Networks (2017)
Deep Quaternion
Networks (2017)
P-Adic
Neural Networks (2004)
Deep
...
- 188k
8
votes
Accepted
Difference between deep neural networks and expectation maximization algorithm
A high level view for neural networks is that they are just heuristics for approximating multi-variate functions. You pick a class of parametrizable functions and you optimize their parameters. ...
- 330
8
votes
Theoretical results on neural networks
Your question is a bit too broad, but here is something you may want to read, if you are interested in the Mathematical Analysis of Deep Learning: The Modern Mathematics of Deep Learning, by Berner et ...
7
votes
Abstract mathematical concepts/tools appeared in machine learning research
Probably, one the most striking is the "UMAP" (Uniform manifold approximation and projection) - a method of dimensional reduction in machine learning. The authors of the method use CATEGORY THEORY ...
7
votes
Mathematics for machine learning
I believe Ian Goodfellow and Yoshua Bengio's Deep Learning book covers the basics and also how you would use it for research. The chapters are also available online for free.
- 179
7
votes
Deep learning / Deep neural nets for mathematician
Since this question got bumped up to the front page somehow, I'm taking the liberty to suggest a partial introduction to the "Math of Deep Learning" given in the following article: The ...
7
votes
Using a poset or directed graph as input for a neural network
This is most definitely NOT the right spot to ask such a question, thought it is a good one.
I happen to dabble precisely with these things in these days, for my own work and research, so I think I ...
- 7,853
7
votes
Theoretical results on neural networks
ICLR 2021 has contributions that could qualify as "rigorous results", one you may like is Minimum Width for Universal Approximation.
The universal approximation property of width-bounded ...
6
votes
Theoretical results on neural networks
The Representer Theorem by Michael Unser has recently unveiled explicit connections between deep NNs (using ReLUs as nonlinearities I believe) and splines.
One core idea is that both the linear ...
5
votes
Deep learning / Deep neural nets for mathematician
Note: The following is an answer to this post but everything I posted there applies equally and fully here.
Let me also comment shortly, that I also found the theory of DNNs difficult to enter since a ...
5
votes
Deep learning / Deep neural nets for mathematician
I just found this paper-https://arxiv.org/pdf/1801.05894.pdf, which introduces deep learning in a mathematically sound manner, specially for computations of backpropagation etc. As a mathematician who ...
4
votes
Universal approximation theorem for whole $\mathbb{R}^d$
I do not think a universal approximation theorem on all of $\mathbb{R}^d$ is possible with the uniform norm. In $L^p$ for $p < \infty$ there may be hope in some cases.
Let us first look at the ...
- 141
4
votes
Mathematics for machine learning
As a deep learning practitioner with mathematical background I was yearning to have some satisfying mathematical framework of what I do in my every day job. In my opinion, very well fitted ...
- 1,615
4
votes
Deep learning / Deep neural nets for mathematician
There's an ongoing course taught by Elchanan Mossel at MIT that you might find helpful. It really focuses on the things we can actually prove about deep learning, which may be mathematically appealing ...
3
votes
Deep learning / Deep neural nets for mathematician
Eldad Haber at the University of British Columbia is exploring connections between neural networks, and dynamical systems:
(2018) Deep Neural Networks Motivated by Partial Differential Equations
(...
2
votes
Uniform Lipschitz function approximation by shallow neural networks
Maybe you can check Theorem 4 in Poggio et al. "Why and When Can Deep-but Not Shallow-Networks Avoid the Curse of Dimensionality: A Review"
- 21
2
votes
Deep learning / Deep neural nets for mathematician
I'd say that deep learning (from a mathematician's perspective) is a HOT MESS. People are jumping through hoops trying to increase accuracy (sometimes just by decimals) introducing all sorts of stuff ...
2
votes
Deep learning / Deep neural nets for mathematician
The original question was asked in 2015. So I believe it is appropriate to include surveys with mathematical flavor on more recent/advanced topics in neural networks and deep learning.
A Mathematical ...
1
vote
How sensitive are Neural Networks to weight change?
Output of the ReLU network is
$$v = \sum_{ij} X_i A_{ij} w^{(1)}_{ij} \cdots w^{(L)}_{ij} $$
where $i$ is the input $j$ is the path and $A_{ij}$ is $1$ if the path is open and $0$ otherwise. Now if ...
1
vote
Mathematics for machine learning
I recently wrote an answer to a related question, about math research that can enhance machine learning. But, part of what I wrote is also related to a learning resource that might help someone ...
- 30.3k
Only top scored, non community-wiki answers of a minimum length are eligible