# Tag Info

## Hot answers tagged machine-learning

51

There are two principles at play here: a mathematical principle that favors hexagonal networks, and a physical principle that favors a network with straight walls. The mathematical principle that prefers hexagonal planar networks is Euler's theorem applied to the two-torus $\mathbb{T}^2$ (to avoid boundary effects), $$V-E+F=0,$$ with $V$ the number of ...

45

Essential elements$^*$ of the reconstruction algorithm were developed at MIT under the name CHIRP = Continuous High-resolution Image Reconstruction using Patch priors, as described in Computational Imaging for VLBI Image Reconstruction (2015). The difficulty of VLBI (Very Long Baseline Interferometry Image) reconstruction is that the inversion problem is ...

24

There is this theorem of Thomas Hales from 1999, which proves the Honeycomb Conjecture: Theorem. Let $\Gamma$ be a locally finite graph in $\mathbb{R}^2$, consisting of smooth curves, and such that $\mathbb{R}^2\setminus \Gamma$ has infinitely many bounded connected components, all of unit area. Let $C$ be the union of these bounded components. Then $$\... 22 Is the following any clearer? (Read the subscripts carefully!)$$\begin{array}{ll} f_1(y_1) + f_2(y_2) + f_3(y_3) + \cdots + f_n(y_n) &\leq \\ f_1(y_2) + f_2(y_2) + f_3(y_3) + \cdots + f_n(y_n) & \leq \\ f_1(y_3) + f_2(y_3) + f_3(y_3) + \cdots + f_n(y_n) & \leq \\ \qquad \qquad \qquad \vdots & \leq \\ f_1(y_n) + f_2(y_n) + f_3(y_n) + \...

19

The answer is no by a Cantor diagonal argument: Let $\Omega=(0,1)$. Let $G$ be all functions that can be computed by a finite number of registers with finite precision. It does not matter where $G$ is learnt from. The number of states of $n$ registers with precision $m$ is finite, thus the number of functions computable on $n$ registers with precision $m$ ...

18

A "roadmap type" introduction is given by Roger Grosse in Differential geometry for machine learning. Differential geometry is all about constructing things which are independent of the representation. You treat the space of objects (e.g. distributions) as a manifold, and describe your algorithm in terms of things that are intrinsic to the manifold ...

14

You can read this paper: https://arxiv.org/abs/1805.10451 Although what they say is by no means new - Coifman had had this point of view for the last thirty years at least.

13

I suggest taking a look at the work of Ulf Grenander. His 1963 book laid out the basis for applying probability theory to groups (Chapter 4 is on stochastic Lie groups) and other algebraic structures. He continued to develop these ideas (see his later book) in the context of pattern recognition. There is definitely newer work in this area (some of which ...

12

Here are some of the group theoretical references within the machine learning literature: Have a look at recent papers by Stéphane Mallat , or first look at 2. This NIPS 2012 talk by Stéphane Mallat A Group Theoretic perspective of Deep Learning Some papers by Risi Kondor, and also his thesis ("Group theoretical methods in machine learning") Directional ...

12

The purely number-theoretic problem can be stated as: For every $n$, are there reals $b$ and $c_i$ such that $$j\text{ is prime iff }\sum_{i=2}^n c_i \gcd(i,j)>b\ ?$$ This is the same as the version above, after removing the $x$'s and $y$'s, shifting the indices by 1, and incorporating a factor of $y_i$ into the $c_i$. For example, for $n=6$, this is ...

12

The answer is NO from general no-free-lunch principles. In particular, the collection of all continuous functions has infinite fat-shattering dimension, and hence is not learnable in your sense. See Alon, Ben-David, Cesa-Bianchi, and Haussler - Scale-sensitive dimensions, uniform convergence, and learnability.

11

The question as asked is rather broad, because there are several works in ML/AI dedicated to mixing time analysis, as well as to detecting if mixing has happened. I would not draw too sharp a boundary about whether the work is in the ML domain or in a closely related domain. Though, I agree, in ML, often MCMC is used with Bayesian methods, and given the ...

10

Isn't it just the 2d sphere packing? If one assumes that the larvae needs a disc of fixed radius to grow up to an adult form and that the bees want to have as many cells as possible then the hexagonal lattice is the optimal one.

9

Mathematics of Deep Learning (2017) This tutorial will review recent work that aims to provide a mathematical justification for several properties of deep networks, such as global optimality, geometric stability, and invariance of the learned representations. Deep Learning: An Introduction for Applied Mathematicians (2018) This article provides ...

9

Here is a classic article by L. Fejes Toth on this subject. https://projecteuclid.org/euclid.bams/1183526078

9

Here is a paragraph of THE LIFE OF THE BEE (1901) By Maurice Maeterlinck: "There are only," says Dr. Reid, "three possible figures of the cells which can make them all equal and similar, without any useless interstices. These are the equilateral triangle, the square, and the regular hexagon. Mathematicians know that there is not a fourth way ...

8

"I am looking for some simple concrete examples of the ways in which real problems go through graph signal processing and how graph Fourier transforms are obtained." • A concrete example of a graph Fourier transform, to the Minnesota road network, is presented in Fourier Analysis on Graphs; another example, to genetic profiling for cancer subtype ...

8

As to question 2, there are certainly plenty of non-trivial discrete models in statistical physics, such as the Ising or Potts models, or lattice gauge theories with discrete gauge groups, that require no partial derivatives (or indeed any operations of differential calculus) at all to formulate and simulate. Similarly, quantum mechanics can be formulated ...

7

Automatic differentiation needs the structure of the function ( computation graph, or preferably a straight line program). In your case, I am not sure how numeric differentiation helps to get a reliable result. If your parameter space is high-dimensional, you are completely screwed. If not, you can interpolate the function by a smooth function (...

7

Topological data analysis is already pretty popular in "data science", so much so that there are companies built on this idea. See: 1). https://en.wikipedia.org/wiki/Topological_data_analysis 2).https://web.stanford.edu/class/archive/ee/ee392n/ee392n.1146/lecture/may13/EE392n_TDA_online.pdf 3). https://towardsdatascience.com/a-concrete-application-of-...

6

Taco Cohen has written some papers that use Differential Geometry, Topology, Guage theory, etc. in Machine learning.

6

Well if you take out partial derivatives, at least quantum field theory and in particular conformal field theory will survive the massacre. The reason is explained in my MO answer: $p$-adic numbers in physics One can use random/quantum fields $\phi:\mathbb{Q}_{p}^{d}\rightarrow \mathbb{R}$ as toy models of fields $\phi:\mathbb{R}^d\rightarrow\mathbb{R}$. In ...

5


5

Mark Jerrum has rigorous results (bounds) on mixing times for Markov Chain Monte Carlo algorithms, as summarized in this presentation. In the context of deep learning, such bounds have been used in Layerwise Systematic Scan: Deep Boltzmann Machines and Beyond.

5

Yet, some more relations of group theory to machine learning: From "Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network Risi Kondor, Zhen Lin, Shubhendu Trivedi": Recent work by Cohen \emph{et al.} has achieved state-of-the-art results for learning spherical images in a rotation invariant way by using ideas from group ...

5

Is it possible to accurately simulate any non-trivial physics without computing partial derivatives? Yes. An example is the nuclear shell model as formulated by Maria Goeppert Mayer in the 1950's. (The same would also apply to, for example, the interacting boson model.) The way this type of shell model works is that you take a nucleus that is close to a ...

4

In the following, I assume a zero mean function, for simplicity. The realizations of a stationary Gaussian process are (a.s.) $n$-times differentiable if the covariance function $k(t_1-t_2):=K(t_1,t_2)$ is $2n$-times differentiable at 0. Hence, in the following, I assume the Gaussian radial basis function $$k(t) = \exp\left(-\frac{1}{2} t^2\right)$$ as ...

4

If your $f$ is a probability distribution, then you can use a kernel density estimate to estimate the derivative. For a bit more detail and relevant references, see section 2.2 of A Tutorial on Kernel Density Estimation and Recent Advances by Yen-Chi Chen. Another approach that might work better in high dimensions would be to use a neural net to get an ...

4

The variation of information seems to be the sort of thing you're looking for.

4

Let $x_1, \dots, x_8$ be $8$ points on a circle in this order. Then there is no triangle containing exactly $x_2, x_4, x_6, x_8$ in its interior and the other four points in its exterior. Intuitively, it would have to intersect the circle in at least $8$ points, but any triangle intersects a circle in at most $6$ points.

Only top voted, non community-wiki answers of a minimum length are eligible