I am interested in knowing about abstract mathematical concepts, tools or methods that have come up in theoretical machine learning. By "abstract" I mean something that is not immediately related to that realm. For instance, a concept from mathematical optimization does not qualify since optimization is directly related to the training of deep networks. In contrast, to me Topological Data Analysis is a non-trivial example of applying algebraic topology to data analysis.

Here are few examples that I have encountered in the literature (all in the context of deep learning).

1. Betti numbers have been utilized to introduce a complexity measure which could be used for comparing deep and shallow architectures:
   https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2014-44.pdf
2. A connection between Sharkovsky's Theorem and the expressive power of deep networks: https://arxiv.org/pdf/1912.04378.pdf
3. An application of Riemannian geometry:
   https://arxiv.org/pdf/1606.05340.pdf
4. Algebraic geometry naturally comes up in studying neural networks with polynomial activation functions. This paper discusses functional varieties associated with such networks: https://arxiv.org/abs/1905.12207

I find it useful to compile a list of such research works on ML that draw on pure math.