11
$\begingroup$

I am interested in knowing about abstract mathematical concepts, tools or methods that have come up in theoretical machine learning. By "abstract" I mean something that is not immediately related to that realm. For instance, a concept from mathematical optimization does not qualify since optimization is directly related to the training of deep networks. In contrast, to me Topological Data Analysis is a non-trivial example of applying algebraic topology to data analysis.

Here are few examples that I have encountered in the literature (all in the context of deep learning).

  1. Betti numbers have been utilized to introduce a complexity measure which could be used for comparing deep and shallow architectures:
    https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2014-44.pdf
  2. A connection between Sharkovsky's Theorem and the expressive power of deep networks: https://arxiv.org/pdf/1912.04378.pdf
  3. An application of Riemannian geometry:
    https://arxiv.org/pdf/1606.05340.pdf
  4. Algebraic geometry naturally comes up in studying neural networks with polynomial activation functions. This paper discusses functional varieties associated with such networks: https://arxiv.org/abs/1905.12207

I find it useful to compile a list of such research works on ML that draw on pure math.

Improve this question
$\endgroup$
11
  • $\begingroup$ Artificial Intelligence SE and Theoretical Computer Science SE are probably better places to ask this question than here. $\endgroup$
    – user156472
    Commented Apr 18, 2020 at 0:11
  • $\begingroup$ @nbro it seems to me that the question is math-focussed. $\endgroup$
    – YCor
    Commented Apr 18, 2020 at 0:31
  • $\begingroup$ @YCor Yes, but learning theory is a central topic in computer science and artificial intelligence. But I agree with you that this question may not be off-topic here. I just think that those other sites may also be appropriate (if not more appropriate) to ask this question. $\endgroup$
    – user156472
    Commented Apr 18, 2020 at 0:33
  • 1
    $\begingroup$ Hard to answer it since every example you gave arises as 'naturally' in machine learning as optimization does. $\endgroup$
    – Piyush Grover
    Commented Apr 18, 2020 at 0:33
  • $\begingroup$ Possibly relevant mathoverflow.net/questions/266028/… $\endgroup$
    – YCor
    Commented Apr 18, 2020 at 0:38

1 Answer 1

Reset to default
7
$\begingroup$

Probably, one the most striking is the "UMAP" (Uniform manifold approximation and projection) - a method of dimensional reduction in machine learning. The authors of the method use CATEGORY THEORY for its discovery. Well, there are certain discussions to what extent category theory is really required, see John Baez blog and references there in, still it is the author's original viewpoint how the method has been discovered. (The algorithm/implementation can be understood without category theory).

The method become quite quickly very popular - gaining 748 citations in two years according to google scholar. And found applications in many fields including bioinformatics (UMAP Nature), as well as capable to produce beautiful images MO355631.

It is similar to previously widely used method - tSNE (T-distributed stochastic neighbor embedding), however, quite often produces better results with less computational efforts, thus beating the predecessor both in quality and speed.

The documentations can be found here: UMAP docs.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .