Questions tagged [machine-learning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
92 votes
13 answers
14k views

Deep learning / Deep neural nets for mathematician

I am interested in finding out the math ideas behind the technologies that are under the umbrella of "Deep Learning" or "Deep neural nets". Most of the papers/books that are often quoted in papers/...
90 votes
11 answers
13k views

What are possible applications of deep learning to research mathematics?

With no doubt everyone here has heard of deep learning, even if they don't know what it is or what it is good for. I myself am a former mathematician turned data scientist who is quite interested in ...
56 votes
10 answers
7k views

A clear map of mathematical approaches to Artificial Intelligence

I have recently become interested in Machine Learning and AI as a student of theoretical physics and mathematics, and have gone through some of the recommended resources dealing with statistical ...
56 votes
4 answers
13k views

Group theory in machine learning

I'm a Machine Learning researcher who would like to research applications of group theory in ML. There is a term "Partially Observed Groups" in machine learning theory which has been ...
drosophyllum's user avatar
52 votes
5 answers
8k views

Why do bees create hexagonal cells ? (Mathematical reasons)

Question 0 Are there any mathematical phenomena which are related to the form of honeycomb cells? Question 1 Maybe hexagonal lattices satisfy certain optimality condition(s) which are related to it? ...
Alexander Chervov's user avatar
52 votes
1 answer
5k views

Mathematics of imaging the black hole

The first ever black hole was "pictured" recently, per an announcement made on 10th April, 2019. See for example: https://www.bbc.com/news/science-environment-47873592 . It has been claimed that ...
Piyush Grover's user avatar
50 votes
1 answer
8k views

What mathematical problems can be attacked using DeepMind's recent mathematical breakthroughs?

I am a research mathematician at a university in the United States. My training is in pure mathematics (geometry). However, for the past couple of months, I have been supervising some computer science ...
Ryan Hendricks's user avatar
36 votes
8 answers
17k views

How useful is differential geometry and topology to deep learning?

After seeing this article https://www.quantamagazine.org/an-idea-from-physics-helps-ai-see-in-higher-dimensions-20200109/ I wanted to ask myself how useful of an endeavor would it be if one goes ...
VS.'s user avatar
  • 1,816
28 votes
6 answers
10k views

Mathematics for machine learning

I would like to know what mathematics topics are the most important to learn before actually studying the theory on neural networks. I ask that because I will start to learn about neural networks and ...
marcosdecarvalho's user avatar
28 votes
1 answer
4k views

Is there any paper which summarizes the mathematical foundation of deep learning?

Is there any paper which summarizes the mathematical foundation of deep learning? Now, I am studying about the mathematical background of deep learning. However, unfortunately I cannot know to what ...
almnagako's user avatar
  • 341
28 votes
1 answer
2k views

Conjectures inspired by AI

Today in Nature a paper described how AI guided mathematicians to make highly non-trivial conjectures, which they managed to prove, one in Knot Theory involving a new invariant, the other in ...
23 votes
1 answer
4k views

Relation between information geometry and geometric deep learning

Disclaimer: This is a cross-post from a very similar question on math.SE. I allowed myself to post it here after reading this meta post about cross-posting between mathoverflow and math.SE, I did ...
Blupon's user avatar
  • 333
22 votes
4 answers
2k views

Open problems which might benefit from computational experiments

Question: I wonder what are the open problems , where computational experiments might me helpful? (Setting some bounds, excluding some cases, shaping some expectations ). Grant program: The context of ...
20 votes
3 answers
3k views

How can Machine Learning help “see” in higher dimensions?

The news that DeepMind had helped mathematicians in research (one in representation theory, and one in knot theory) certainly got many thinking, what other projects could AI help us with? See MO ...
liuyao's user avatar
  • 485
18 votes
0 answers
295 views

Profiles of very high dimensional functions

This question comes from trying to understand the recent success of deep neural nets. Neural networks just (crudely speaking) create a very complicated function of very many variables, and then ...
Igor Rivin's user avatar
  • 95.5k
17 votes
3 answers
1k views

Theoretical results on neural networks

With this question I'd like to have a recollection of theoretical rigorous results on neural networks. I'd like to have results that have been settled, as opposed to hypothesis. As an example, this ...
16 votes
1 answer
825 views

Are primes linearly separable?

Let $X_1,\cdots,X_n$ be finite subsets of some set $Z$. Then the symmetric difference metric space: $$d(X_i,X_j) = \sqrt{ |X_i|+|X_j|-2|X_i\cap X_j|}$$ can be embedded in Euclidean space. The value $|...
user avatar
16 votes
2 answers
1k views

Physical interpretation of the Manifold Hypothesis

Motivation: Most dimensionality reduction algorithms assume that the input data are sampled from a manifold $\mathcal{M}$ whose intrinsic dimension $d$ is much smaller than the ambient dimension $D$. ...
Aidan Rocke's user avatar
  • 3,827
15 votes
3 answers
773 views

Neural networks over gadgets other than $\mathbb{R}$

Recently, I learned that neural networks (NN) can be defined over fields other than $\mathbb{R}$: for example, Khrennikov and Tirozzi wrote a paper in 1999 (!) on $p$-adic neural networks, or neural ...
xuq01's user avatar
  • 1,054
14 votes
6 answers
3k views

Mathematical physics without partial derivatives

Remark: All the answers so far have been very insightful and on point but after receiving public and private feedback from other mathematicians on the MathOverflow I decided to clarify a few notions ...
Aidan Rocke's user avatar
  • 3,827
13 votes
1 answer
2k views

Various authors of the Bourbaki's books

As far as I understand, each chapter of the Bourbaki's collection was written by one (or two?) specific authors. The book itself was reviewed, corrected and after all approved by the whole Bourbaki ...
Libli's user avatar
  • 7,210
13 votes
2 answers
665 views

Reference Request: Theoretical Mixing Times Research in Machine Learning / Artificial Intelligence (AI)

I'm doing a PhD in probability theory, focusing mostly on mixing times. It's a pure maths PhD, considering precise models and showing rigorous mixing results. I'm also interested in stuff like machine ...
Sam OT's user avatar
  • 560
12 votes
2 answers
2k views

Is it ever unnecessary to mathematically formalize a concept?

From my understanding, mathematics sometimes gives rise to new physical/tangible laws and the converse is also true. In particular, physical phenomena give rise to new mathematics. In all of the cases ...
Alginpeter's user avatar
11 votes
2 answers
1k views

Soft question: Deep learning and higher categories

Recently, I have stumbled upon certain articles and lecture videos that use category theory to explain certain aspects of machine learning or deep learning (e.g. Cats for AI and the paper An enriched ...
h3fr43nd's user avatar
  • 221
11 votes
1 answer
656 views

Abstract mathematical concepts/tools appeared in machine learning research

I am interested in knowing about abstract mathematical concepts, tools or methods that have come up in theoretical machine learning. By "abstract" I mean something that is not immediately related to ...
10 votes
3 answers
1k views

Deep learning for knot theory. Classification

As far as I know, there is a classification of all prime knots with less than 16 crossings. It seems that there is already a fast enough algorithm to distinguish a knot from an unknot. So in principle ...
GSM's user avatar
  • 343
9 votes
1 answer
2k views

Is There an Induction-Free Proof of the 'Be The Leader' Lemma?

This lemma is used in the context of online convex optimisation. It is sometimes called the 'Be the Leader Lemma'. Lemma: Suppose $f_1,f_2,\ldots,f_N$ are real valued functions with the same ...
Daron's user avatar
  • 1,761
9 votes
1 answer
293 views

Who introduced the term hyperparameter?

I am trying to find the earliest use of the term hyperparameter. Currently, it is used in machine learning but it must have had earlier uses in statistics or optimization theory. Even the multivolume ...
AChem's user avatar
  • 803
9 votes
1 answer
660 views

Sufficient condition for linear separability of a boolean function on $n$ variables

This is a cross-post of two recent questions at math.stackexchange without answers: Q1 and Q2. A boolean function on an $n$-dimensional hypercube is linearly separable when the convex hulls of the ...
Fabius Wiesner's user avatar
9 votes
0 answers
130 views

Worst margin when halving a hypercube with a hyperplane

Consider the $n$-cube $C_n=\lbrace-1,1\rbrace^n$ and the problem of partitioning it into halves with hyperplanes through the origin that avoid all its points. We can parameterize the hyperplanes by ...
Veit Elser's user avatar
  • 1,045
9 votes
0 answers
258 views

How sensitive are Neural Networks to weight change?

Let's consider the space of feedforward neural networks with a given structure: $L$ layers, $m$ neurones per layer, ReLu activation, input dimension $d$, output dimension $k$. Which means I'm ...
Alfred's user avatar
  • 879
8 votes
4 answers
2k views

How to learn a continuous function?

Let $\Omega \subset \mathbb{R}^m$ be an open subset bounded with a smooth boundary. Problem : Given any bounded continuous function $f:\Omega\to\mathbb{R}$, can we learn it to a given accuracy $\...
Rajesh D's user avatar
  • 704
8 votes
1 answer
518 views

Geometric formulation of the subject of machine learning

Question: what is the geometric interpretation of the subject of machine learning and/or deep learning? Being "forced" to have a closer look at the subject, I have the impression that it ...
Manfred Weis's user avatar
  • 12.6k
8 votes
1 answer
2k views

graph signal processing

I have read this article https://arxiv.org/abs/1307.5708 about vertix-frequency analysis on graph. David IShuman in this article claims that,"we generalize one of the most important signal ...
niloofar jamshidi's user avatar
8 votes
2 answers
937 views

Approximation of Wasserstein distance between $p_\theta$ and $p_{\theta + d\theta}$

Given a parametric family of distributions $\{p_\theta\mid\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid $$\operatorname{KL}(p_\theta\...
dohmatob's user avatar
  • 6,716
8 votes
2 answers
1k views

VC dimension, fat-shattering dimension, and other complexity measures, of a class BV functions

I wish to show that a function which is "essentially constant" (defined shortly) can't be a good classifier (machine learning). For this i need to estimate the "complexity" of such a class of ...
dohmatob's user avatar
  • 6,716
8 votes
0 answers
307 views

Monotonicity of log determinant of Gaussian kernel matrix

Let \begin{equation} k({x},{y}) = \sigma \exp\left(-\frac{(x-y)^2}{2\theta^2}\right)\end{equation} be a squared-exponential (Gaussian) kernel, with $\sigma,\vartheta>0$. Consider, for a set of $N$ ...
Heinrich A's user avatar
8 votes
0 answers
118 views

Positive definite kernels on categories

I'm wondering if there is any work on studying positive definite kernels on (the objects of a) category. By this I mean for a category $\mathcal{C}$, find a function $$ K: Ob\mathcal{C} \times Ob\...
Eric's user avatar
  • 855
7 votes
2 answers
1k views

Mathematics of GANs (generative adversarial networks)

Generative Adversarial Networks were introduced in http://papers.nips.cc/paper/5423-generative-adversarial-nets and has more than 20000 citations. The paper introduced key paradigm changes which ...
Turbo's user avatar
  • 13.7k
7 votes
2 answers
393 views

Upper bound on VC-dimension of partitioned class

Fix $n,k\in \mathbb{N}_+$. Let $\mathcal{H}$ be a set of functions from $\mathbb{R}^n$ to $\mathbb{R}$ with finite VC-dimension $d\in \mathbb{N}$. Let $\mathcal{H}_k$ denote the set of maps of the ...
Math_Newbie's user avatar
7 votes
1 answer
396 views

Does the plane clustered to minimize sum distances^2 to clusters centers ( inertia / "K-means") produce hexagonal clusters / hexagonal lattice?

"K-means" is the most simple and famous clustering algorithm, which has numerous applications. For a given as an input number of clusters it segments set of points in R^n to that given number of ...
Alexander Chervov's user avatar
7 votes
1 answer
484 views

Books to develop a unified view of statistics and information theory?

I hope to understand the connection between statistics and information theory in a deep philosophical sense. I suppose the best place to start would be David MacKay's Information Theory, Inference, ...
inhaler18's user avatar
7 votes
1 answer
303 views

Iterating projections to random halfspaces

Consider the following process: Start with a set $S = \mathbb R^n$. Repeat $L$ times: choose a random orthonormal basis $u_1, \ldots, u_n$, and consider the cone $C = \{ \sum \alpha_i u_i : \alpha_i \...
Daniel Paleka's user avatar
6 votes
2 answers
3k views

Derivatives of Gaussian processes

A Gaussian process $X$ on Euclidean space $\mathbb R^d$ has a radial basis kernel if for any $u,w\in\mathbb R^d$, we have $$ \mathrm{Cov}(X_u, X_w) = \sigma^2 \exp\left ( -\frac{\left\lVert u-w \right ...
Shannon S.'s user avatar
6 votes
1 answer
1k views

Why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?

I'm reading 《Algebraic geometry and statistical learning theory》.My problem is why the Fisher information matrix is equal to the Hessian matrix of the Kullback–Leibler distance at the true parameter?...
phybrain's user avatar
  • 103
6 votes
1 answer
3k views

Covering number of Lipschitz functions

What do we know about the covering number of $L$-Lipschitz functions mapping say, $\mathbb{R}^n \rightarrow \mathbb{R}$ for some $L >0$? Only 2 results I have found so far are, That the $\infty$-...
gradstudent's user avatar
  • 2,136
6 votes
1 answer
821 views

Universal approximation theorem for whole $\mathbb{R}^d$

The well-known universal approximation theorem states that neural network with one hidden layer can approximate any continuous function on every compact subset of $\mathbb{R}^d$. My question is ...
almnagako's user avatar
  • 341
6 votes
0 answers
270 views

Mathematical questions or areas amenable to AI [duplicate]

This question regards the new paper "Advancing mathematics by guiding human intuition with AI" by Davies et al. (Nature, 2021) (DOI link in open access) in which researchers at Deepmind ...
6 votes
0 answers
327 views

Signature and cusp geometry of hyperbolic knots

Nature recently published a paper titled “Advancing mathematics by guiding human intuition with AI”. Using the power of linear algebra and calculus machine learning, the authors link "geometric&...
Steve's user avatar
  • 2,223
5 votes
3 answers
2k views

Automatic vs numerical differentiation of a function known from samples

Suppose I have $n$ samples $(x_i, f(x_i))_{i=1}^n$ from an unknown function $f$. I need to approximate (estimate) the derivative $f'(x^*)$ at some new test point $x^*$, that is not necessarily one of ...
JohnA's user avatar
  • 680