Questions tagged [machine-learning]
The machine-learning tag has no usage guidance.
178
questions
92
votes
13
answers
14k
views
Deep learning / Deep neural nets for mathematician
I am interested in finding out the math ideas behind the technologies that are under the umbrella of "Deep Learning" or "Deep neural nets".
Most of the papers/books that are often quoted in papers/...
90
votes
11
answers
13k
views
What are possible applications of deep learning to research mathematics?
With no doubt everyone here has heard of deep learning, even if they don't know what it is or what it is good for. I myself am a former mathematician turned data scientist who is quite interested in ...
56
votes
10
answers
7k
views
A clear map of mathematical approaches to Artificial Intelligence
I have recently become interested in Machine Learning and AI as a student of theoretical physics and mathematics, and have gone through some of the recommended resources dealing with statistical ...
56
votes
4
answers
13k
views
Group theory in machine learning
I'm a Machine Learning researcher who would like to research applications of group theory in ML.
There is a term "Partially Observed Groups" in machine learning theory which has been ...
52
votes
5
answers
8k
views
Why do bees create hexagonal cells ? (Mathematical reasons)
Question 0 Are there any mathematical phenomena which are related to the form of honeycomb cells?
Question 1 Maybe hexagonal lattices satisfy certain optimality condition(s) which are related to it? ...
52
votes
1
answer
5k
views
Mathematics of imaging the black hole
The first ever black hole was "pictured" recently, per an announcement made on 10th April, 2019. See for example: https://www.bbc.com/news/science-environment-47873592 .
It has been claimed that ...
50
votes
1
answer
8k
views
What mathematical problems can be attacked using DeepMind's recent mathematical breakthroughs?
I am a research mathematician at a university in the United States. My training is in pure mathematics (geometry). However, for the past couple of months, I have been supervising some computer science ...
36
votes
8
answers
17k
views
How useful is differential geometry and topology to deep learning?
After seeing this article https://www.quantamagazine.org/an-idea-from-physics-helps-ai-see-in-higher-dimensions-20200109/ I wanted to ask myself how useful of an endeavor would it be if one goes ...
28
votes
6
answers
10k
views
Mathematics for machine learning
I would like to know what mathematics topics are the most important to learn before actually studying the theory on neural networks.
I ask that because I will start to learn about neural networks and ...
28
votes
1
answer
4k
views
Is there any paper which summarizes the mathematical foundation of deep learning?
Is there any paper which summarizes the mathematical foundation of deep learning?
Now, I am studying about the mathematical background of deep learning.
However, unfortunately I cannot know to what ...
28
votes
1
answer
2k
views
Conjectures inspired by AI
Today in Nature a paper described how AI guided mathematicians to make highly non-trivial conjectures, which they managed to prove, one in Knot Theory involving a new invariant, the other in ...
23
votes
1
answer
4k
views
Relation between information geometry and geometric deep learning
Disclaimer: This is a cross-post from a very similar question on math.SE. I allowed myself to post it here after reading this meta
post about cross-posting between mathoverflow and math.SE, I did
...
22
votes
4
answers
2k
views
Open problems which might benefit from computational experiments
Question: I wonder what are the open problems , where computational experiments might me helpful? (Setting some bounds, excluding some cases, shaping some expectations ).
Grant program: The context of ...
20
votes
3
answers
3k
views
How can Machine Learning help “see” in higher dimensions?
The news that DeepMind had helped mathematicians in research (one in representation theory, and one in knot theory) certainly got many thinking, what other projects could AI help us with? See MO ...
18
votes
0
answers
295
views
Profiles of very high dimensional functions
This question comes from trying to understand the recent success of deep neural nets. Neural networks just (crudely speaking) create a very complicated function of very many variables, and then ...
17
votes
3
answers
1k
views
Theoretical results on neural networks
With this question I'd like to have a recollection of theoretical rigorous results on neural networks.
I'd like to have results that have been settled, as opposed to hypothesis. As an example, this ...
16
votes
1
answer
825
views
Are primes linearly separable?
Let $X_1,\cdots,X_n$ be finite subsets of some set $Z$. Then the symmetric difference metric space:
$$d(X_i,X_j) = \sqrt{ |X_i|+|X_j|-2|X_i\cap X_j|}$$
can be embedded in Euclidean space. The value $|...
16
votes
2
answers
1k
views
Physical interpretation of the Manifold Hypothesis
Motivation:
Most dimensionality reduction algorithms assume that the input data are sampled from a manifold $\mathcal{M}$ whose intrinsic dimension $d$ is much smaller than the ambient dimension $D$. ...
15
votes
3
answers
773
views
Neural networks over gadgets other than $\mathbb{R}$
Recently, I learned that neural networks (NN) can be defined over fields other than $\mathbb{R}$: for example, Khrennikov and Tirozzi wrote a paper in 1999 (!) on $p$-adic neural networks, or neural ...
14
votes
6
answers
3k
views
Mathematical physics without partial derivatives
Remark: All the answers so far have been very insightful and on point but after receiving public and private feedback from other mathematicians on the MathOverflow I decided to clarify a few notions ...
13
votes
1
answer
2k
views
Various authors of the Bourbaki's books
As far as I understand, each chapter of the Bourbaki's collection was written by one (or two?) specific authors. The book itself was reviewed, corrected and after all approved by the whole Bourbaki ...
13
votes
2
answers
665
views
Reference Request: Theoretical Mixing Times Research in Machine Learning / Artificial Intelligence (AI)
I'm doing a PhD in probability theory, focusing mostly on mixing times. It's a pure maths PhD, considering precise models and showing rigorous mixing results. I'm also interested in stuff like machine ...
12
votes
2
answers
2k
views
Is it ever unnecessary to mathematically formalize a concept?
From my understanding, mathematics sometimes gives rise to new physical/tangible laws and the converse is also true. In particular, physical phenomena give rise to new mathematics.
In all of the cases ...
11
votes
2
answers
1k
views
Soft question: Deep learning and higher categories
Recently, I have stumbled upon certain articles and lecture videos that use category theory to explain certain aspects of machine learning or deep learning (e.g. Cats for AI and the paper An enriched ...
11
votes
1
answer
656
views
Abstract mathematical concepts/tools appeared in machine learning research
I am interested in knowing about abstract mathematical concepts, tools or methods that have come up in theoretical machine learning. By "abstract" I mean something that is not immediately related to ...
10
votes
3
answers
1k
views
Deep learning for knot theory. Classification
As far as I know, there is a classification of all prime knots with less than 16 crossings.
It seems that there is already a fast enough algorithm to distinguish a knot from an unknot.
So in principle ...
9
votes
1
answer
2k
views
Is There an Induction-Free Proof of the 'Be The Leader' Lemma?
This lemma is used in the context of online convex optimisation. It is sometimes called the 'Be the Leader Lemma'.
Lemma:
Suppose $f_1,f_2,\ldots,f_N$ are real valued functions with the same ...
9
votes
1
answer
293
views
Who introduced the term hyperparameter?
I am trying to find the earliest use of the term hyperparameter. Currently, it is used in machine learning but it must have had earlier uses in statistics or optimization theory. Even the multivolume ...
9
votes
1
answer
660
views
Sufficient condition for linear separability of a boolean function on $n$ variables
This is a cross-post of two recent questions at math.stackexchange without answers: Q1 and Q2.
A boolean function on an $n$-dimensional hypercube is linearly separable when the convex hulls of the ...
9
votes
0
answers
130
views
Worst margin when halving a hypercube with a hyperplane
Consider the $n$-cube $C_n=\lbrace-1,1\rbrace^n$ and the problem of partitioning it into halves with hyperplanes through the origin that avoid all its points. We can parameterize the hyperplanes by ...
9
votes
0
answers
258
views
How sensitive are Neural Networks to weight change?
Let's consider the space of feedforward neural networks with a given structure: $L$ layers, $m$ neurones per layer, ReLu activation, input dimension $d$, output dimension $k$.
Which means I'm ...
8
votes
4
answers
2k
views
How to learn a continuous function?
Let $\Omega \subset \mathbb{R}^m$ be an open subset bounded with a smooth boundary.
Problem : Given any bounded continuous function $f:\Omega\to\mathbb{R}$, can we learn it to a given accuracy $\...
8
votes
1
answer
518
views
Geometric formulation of the subject of machine learning
Question:
what is the geometric interpretation of the subject of machine learning and/or deep learning?
Being "forced" to have a closer look at the subject, I have the impression that it ...
8
votes
1
answer
2k
views
graph signal processing
I have read this article
https://arxiv.org/abs/1307.5708
about vertix-frequency analysis on graph.
David IShuman
in this article claims that,"we generalize one of the most important signal ...
8
votes
2
answers
937
views
Approximation of Wasserstein distance between $p_\theta$ and $p_{\theta + d\theta}$
Given a parametric family of distributions $\{p_\theta\mid\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid
$$\operatorname{KL}(p_\theta\...
8
votes
2
answers
1k
views
VC dimension, fat-shattering dimension, and other complexity measures, of a class BV functions
I wish to show that a function which is "essentially constant" (defined shortly) can't be a good classifier (machine learning). For this i need to estimate the "complexity" of such a class of ...
8
votes
0
answers
307
views
Monotonicity of log determinant of Gaussian kernel matrix
Let \begin{equation} k({x},{y}) = \sigma \exp\left(-\frac{(x-y)^2}{2\theta^2}\right)\end{equation}
be a squared-exponential (Gaussian) kernel, with $\sigma,\vartheta>0$. Consider, for a set of $N$ ...
8
votes
0
answers
118
views
Positive definite kernels on categories
I'm wondering if there is any work on studying positive definite kernels on (the objects of a) category. By this I mean for a category $\mathcal{C}$, find a function
$$
K: Ob\mathcal{C} \times Ob\...
7
votes
2
answers
1k
views
Mathematics of GANs (generative adversarial networks)
Generative Adversarial Networks were introduced in http://papers.nips.cc/paper/5423-generative-adversarial-nets and has more than 20000 citations.
The paper introduced key paradigm changes which ...
7
votes
2
answers
393
views
Upper bound on VC-dimension of partitioned class
Fix $n,k\in \mathbb{N}_+$.
Let $\mathcal{H}$ be a set of functions from $\mathbb{R}^n$ to $\mathbb{R}$ with finite VC-dimension $d\in \mathbb{N}$. Let $\mathcal{H}_k$ denote the set of maps of the ...
7
votes
1
answer
396
views
Does the plane clustered to minimize sum distances^2 to clusters centers ( inertia / "K-means") produce hexagonal clusters / hexagonal lattice?
"K-means" is the most simple and famous clustering algorithm, which has numerous applications.
For a given as an input number of clusters it segments set of points in R^n to that given number of ...
7
votes
1
answer
484
views
Books to develop a unified view of statistics and information theory?
I hope to understand the connection between statistics and information theory in a deep philosophical sense.
I suppose the best place to start would be David MacKay's Information Theory, Inference, ...
7
votes
1
answer
303
views
Iterating projections to random halfspaces
Consider the following process:
Start with a set $S = \mathbb R^n$. Repeat $L$ times: choose a random orthonormal basis $u_1, \ldots, u_n$, and consider the cone $C = \{ \sum \alpha_i u_i : \alpha_i \...
6
votes
2
answers
3k
views
Derivatives of Gaussian processes
A Gaussian process $X$ on Euclidean space $\mathbb R^d$ has a radial basis kernel if for any $u,w\in\mathbb R^d$, we have
$$ \mathrm{Cov}(X_u, X_w) = \sigma^2 \exp\left ( -\frac{\left\lVert u-w \right ...
6
votes
1
answer
1k
views
Why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?
I'm reading 《Algebraic geometry and statistical learning theory》.My problem is why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?...
6
votes
1
answer
3k
views
Covering number of Lipschitz functions
What do we know about the covering number of $L$-Lipschitz functions mapping say, $\mathbb{R}^n \rightarrow \mathbb{R}$ for some $L >0$?
Only 2 results I have found so far are,
That the $\infty$-...
6
votes
1
answer
821
views
Universal approximation theorem for whole $\mathbb{R}^d$
The well-known universal approximation theorem states that neural network with one hidden layer can approximate any continuous function on every compact subset of $\mathbb{R}^d$.
My question is ...
6
votes
0
answers
270
views
Mathematical questions or areas amenable to AI [duplicate]
This question regards the new paper "Advancing mathematics by guiding human intuition with AI" by Davies et al. (Nature, 2021) (DOI link in open access) in which researchers at Deepmind ...
6
votes
0
answers
327
views
Signature and cusp geometry of hyperbolic knots
Nature recently published a paper titled “Advancing mathematics by guiding human intuition with AI”. Using the power of linear algebra and calculus machine learning, the authors link "geometric&...
5
votes
3
answers
2k
views
Automatic vs numerical differentiation of a function known from samples
Suppose I have $n$ samples $(x_i, f(x_i))_{i=1}^n$ from an unknown function $f$. I need to approximate (estimate) the derivative $f'(x^*)$ at some new test point $x^*$, that is not necessarily one of ...