Questions tagged [machine-learning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
65 views

Compression bounds on the Copeland–Erdős constant

Motivation: Given the set of prime numbers $\mathbb{P} \subset \mathbb{N}$, the Copeland–Erdős constant $\mathcal{C}$ is defined as [1]: \begin{equation} \mathcal{C} = \sum_{n=1}^\infty p_n \cdot 10^{-...
0
votes
0answers
19 views

Loss function for matrix with fixed trace

I have a matrix $P \in \mathbb{R}^{n \times m}$ where $n \gg m$ and each row of $P$ has norm $1$. It is not hard to see that the $P^T P$ has a fixed trace $n$. I am try to find a loss function to push ...
0
votes
0answers
25 views

Analytic formula for minimum possible error for functions in RKHS ball, on a simple classificaiton problem

Let $X=\mathcal S_{d-1}$ be the unit-sphere in $\mathbb R^d$, and let $K:X \times X \to \mathbb R^d$ be a Mercer kernel (e.g the Laplace kernel). Let $\mathcal H_K$ be the induced RKHS, and for $R \ge ...
66
votes
9answers
6k views

What are possible applications of deep learning to research mathematics?

With no doubt everyone here has heard of deep learning, even if they don't know what it is or what it is good for. I myself am a former mathematician turned data scientist who is quite interested in ...
0
votes
0answers
72 views

Binary Regression : Is this an open problem in Mathematics/Statistics?

Let $X$ be a random variable which takes values from $\Omega = (0,1)^m$ with a probability distribution $p(x)$. Assume $p$ is a BV function with non zero total variation and $p(x)>0\forall x\in\...
-1
votes
0answers
8 views

Coordinate expansion or multiple regressions

I was wondering if anyone could offer some advice on the most productive direction to head in when seeking to fit a regresion on the below data which is most entirely centered on zero. Currently ...
13
votes
1answer
1k views

Various authors of the Bourbaki's books

As far as I understand, each chapter of the Bourbaki's collection was written by one (or two?) specific authors. The book itself was reviewed, corrected and after all approved by the whole Bourbaki ...
3
votes
1answer
167 views

Ricci flow for manifold learning

I know that mean curvature and diffusion-type flows are common in manifold learning because of their smoothing effects. I haven't seen Ricci flow used as much. Given that Ricci and diffusion-type ...
0
votes
0answers
93 views

Derivative of the function of random variable

Suppose we have a function $\phi(X)$ of random variable $X$. Suppose both of $\phi(X)$ and $X$ are random variables. If $\phi$ is differentiable, how to calculate the derivative of $\phi(X)$ w.r.t. $...
1
vote
1answer
54 views

Bayes risk of binary classification problem with conditionally independet covariates

In the setting of this problem, $\eta(\vec{x})$ is $P(Y=1|\vec{X}=\vec{x})$, $Y \in {0,1}$, $X \in R^d$. Being the true probability know, the classification rule is simply $\eta(\vec{x})>0.5 \...
0
votes
0answers
47 views

UCB1 for Multiarmed bandits: understanding the proof

Regarding Multiarmed Bandits, I am analysing algorithm UCB1 presented in this paper and specifically the proof of Theorem 1, concerning the regret of the algorithm. At page 243 there is a step I do ...
4
votes
1answer
167 views

Iterating projections to random halfspaces

Consider the following process: Start with a set $S = \mathbb R^n$. Repeat $L$ times: choose a random orthonormal basis $u_1, \ldots, u_n$, and consider the cone $C = \{ \sum \alpha_i u_i : \alpha_i \...
3
votes
0answers
148 views

Games and the right mathematical framework for GANs

Generative Adversarial Networks were introduced in http://papers.nips.cc/paper/5423-generative-adversarial-nets and has more than 20000 citations. It is an important topic within deep learning. Are ...
0
votes
0answers
28 views

Bounds on the regularized risk minimizer and risk minimizer?

Some of this is taken from a similar post here, but with modifications. In supervised machine learning, we typically take a Risk Minimization (RM) point of view when formulating a problem. So, what we ...
7
votes
2answers
454 views

Mathematics of GANs (generative adversarial networks)

Generative Adversarial Networks were introduced in http://papers.nips.cc/paper/5423-generative-adversarial-nets and has more than 20000 citations. The paper introduced key paradigm changes which ...
0
votes
1answer
299 views

In linear regression, we have 0 training error if data dimension is high, but are there similar results for other supervised learning problems?

I tried posting this question on Cross Validated (the stack exchange for statistics) but didn't get an answer, so posting here: Let's consider a supervised learning problem where $\{(x_1,y_1) \dots (...
0
votes
1answer
76 views

Finite VC dimension > the number of free parameters

I'm looking for an example of the following: A hypothesis class $\mathcal{H}$ such that $\forall h \in \mathcal{H}$, the number of free parameters of $h$ is equal to $n \in \mathbb{N}$ (where $n < ...
2
votes
0answers
42 views

What are some beginner's references on algebraically structured (statistical) models, and their connection with group actions and Fourier transform?

I asked this question on Cross Validated a few days ago, but didn't really get a favorable response, so asking here to see if I get any. I'm looking at the description of a short-term position in ...
1
vote
0answers
55 views

Relation between minimizer of regularized risk & risk in statistical learning theory

In supervised machine learning, we typically take a Risk Minimization (RM) point of view when formulating a problem. So, what we typically solve for is the following: $$ R^L(h) = \underset{h\in\...
0
votes
1answer
244 views

How large sample $m$ is enough [closed]

I have a $D$ probability distribution over $X =R^d$, i have two samples $s_1$ and $s_2$ from $D$, each having size $m_1$, $m_2$, a unit ball centered at origin $B(0)$, defined by $B(0)=\{x \in R^2: \|...
4
votes
0answers
91 views

Can we show equivalence of two distributions based on their statistics?

Let $p,q$ be two distributions on $\mathbb{R}^d$. Let $f:\mathbb{R}^d\times\mathbb{R}^d\rightarrow\mathbb{R}$. Under what conditions does $\mathbb{E}_{x\sim p}f(x,z)=\mathbb{E}_{x\sim q}f(x,z)\ \...
0
votes
1answer
87 views

Using a poset or directed graph as input for a neural network [closed]

I'm not sure if this is the right community to post this in but I would appreciate any help. As the title states, I'm trying to train a neural network using some unconventional input. I'm wondering if ...
1
vote
0answers
28 views

Restriction of Rademacher Complexity

Let $F\subseteq C([0,1]^n,\mathbb{R})$ be a finite family of functions, which is non-empty. Let $A,B$ be subseteq of $[0,1]^n$, again non-empty, and let $Rad(C)$ denote the Rademacher complexity of ...
1
vote
0answers
52 views

Bayesian inference of stochastically evolving model parameters

I have a question related to self-calibration in radio interferometry, but I will try to phrase it as generic as possible. I have a set of data points, $D = \{ d_{0, t_0}, d_{1, t_0}, ..., d_{M, t_0}, ...
0
votes
0answers
38 views

Relation between test and train error with gradient descent iterates

My question is about establishing an inequality between population error and expected training error (i.e, expected training error < population error) for a model trained with gradient descent on a ...
0
votes
0answers
23 views

parametric model with parameters described as gaussian processes

Let's assume that I have some data $y_{t_i}$ (i = 0, 1, ..., N) and a model $\hat{y}(a(t), b(t))$, where the parameters of my model (a, b) evolve with time t in a stochastic manner. I am wondering if ...
1
vote
1answer
161 views

References for “second order” random walk on graphs (used in “node2vec” paper)?

The "word2vec" family of methods provided a great breakthrough in natural language processing. The methods assign to each word a vector in $R^{n}$, such that the "similar" words ...
5
votes
0answers
94 views

Distance of two points in Grassmannian using Plücker coordinate

Let $G(q,D)$ be the Grassmannian of $q$-dimensional vector spaces in $\mathbb{R}^D$, where $q \le D$ are positive integers. In the paper, a distance between two points of $G(q,D)$ are defined as ...
8
votes
4answers
2k views

How to learn a continuous function?

Let $\Omega \subset \mathbb{R}^m$ be an open subset bounded with a smooth boundary. Problem : Given any bounded continuous function $f:\Omega\to\mathbb{R}$, can we learn it to a given accuracy $\...
3
votes
1answer
74 views

Hermite polynomial after rotation

When we consider the $n$-dimensional standard normal distribution, the orthogonal basis is $\{H_S(x)\}_{S}$ where $H_S(x) = \prod_{k=1}^n H_{s_k}(x_k)$. Here $H_*(x)$ is the normalized probabilist's ...
1
vote
1answer
150 views

Upper bounding VC dimension of an indicator function class

I would like to upper bound the VC dimension of the function class $ F$ defined as follows: $$ F := \left\{ (x,t) \mapsto \mathbb{1} \left( c_Q\min_{q \in Q} {\|x-q \|}_1 - t > 0 \right) \; | \; Q ...
52
votes
5answers
7k views

Why do bees create hexagonal cells ? (Mathematical reasons)

Question 0 Are there any mathematical phenomena which are related to the form of honeycomb cells? Question 1 Maybe hexagonal lattices satisfy certain optimality condition(s) which are related to it? ...
7
votes
1answer
290 views

Does the plane clustered to minimize sum distances^2 to clusters centers ( inertia / “K-means”) produce hexagonal clusters / hexagonal lattice?

"K-means" is the most simple and famous clustering algorithm, which has numerous applications. For a given as an input number of clusters it segments set of points in R^n to that given number of ...
2
votes
1answer
75 views

Uniform Lipschitz function approximation by shallow neural networks

Fix $d\in \mathbb{N}$. Let $F_1$ be the set of all 1-Lipschitz functions mapping $[0, 1]^d$ to $\mathbb{R}$. For $\varphi: \mathbb{R} \rightarrow \mathbb{R}$ and $m \in \mathbb{N}$, let $N_\varphi^m$ ...
0
votes
1answer
97 views

Independence in a sequential problem with observations getting added to buckets

Consider a sequence of random observations $(O(t))_{t\geq 1}$, with $O(t)=(D(t),J(t),Y(t))$. Denote $\mathcal{F}(t) := \sigma(O(1),\ldots,O(t))$, the filtration induced by the first $t$ observations. ...
8
votes
1answer
194 views

Books to develop a unified view of statistics and information theory?

I hope to understand the connection between statistics and information theory in a deep philosophical sense. I suppose the best place to start would be David MacKay's Information Theory, Inference, ...
5
votes
1answer
258 views

Abstract mathematical concepts/tools appeared in machine learning research

I am interested in knowing about abstract mathematical concepts, tools or methods that have come up in theoretical machine learning. By "abstract" I mean something that is not immediately related to ...
2
votes
0answers
126 views

Inequality on the Kullback-Leibler divergence

Let us define the arithmetic, geometric, and harmonic means of $x,y \in \mathbb{R}$ weighted by $\alpha =(\alpha_x,\alpha_y) \in [0,1]$, respectively as \begin{equation} a_\alpha(x,y) = \frac{\...
0
votes
2answers
169 views

Statistical divergence

Does anyone know about a statistical divergence of this type? \begin{equation} \text{D}(P||Q) = \frac{1}{2} \left[\text{KL}(M||P) + \text{KL}(M||Q)\right] \end{equation} where $M = \frac{1}{2} [P+Q]$....
1
vote
0answers
84 views

Comparison of concentrations of different $L^p$-norms of (sub) Gaussian distributions

It's well-known that the Euclidean $2$-norm of subgaussian random vectors concentrates in high dimensions, e.g. when $X \sim \mathcal{N}(0,I_n),$ (or in general $X$ is subgaussian with independent co-...
0
votes
2answers
165 views

Help with a definition of a two-person game in a referenced paper

In the paper "Finding Mixed Nash Equilibria of Generative Adversarial Networks" the authors write in equation (1) on page 2: Consider the classical formulation of a two-player game with finitely ...
1
vote
0answers
37 views

What is the number of iterations needed for the message passing algorithm to converge when applied to an acyclic factor graph?

I understand that the message passing algorithm (Belief Propagation algorithm), when applied to a factor graph consists in an exchange in messages between the factor nodes and the variable nodes, ...
9
votes
4answers
2k views

mathematical physics without partial derivatives

Remark: All the answers so far have been very insightful and on point but after receiving public and private feedback from other mathematicians on the MathOverflow I decided to clarify a few notions ...
2
votes
0answers
55 views

Variational forms of non-convex functions

I am trying to understand what kind of variational forms exist for non-convex functions. Alternatively, are there conjugate forms which attain strong duality? For a non-convex function $f$, I am ...
1
vote
0answers
29 views

sequential learning reference request [closed]

I would like to find a book for master math student about the following topics. I don't know the field and I don't want to be lost in details so if it could be an straight forward please. I'm a ...
7
votes
0answers
99 views

Positive definite kernels on categories

I'm wondering if there is any work on studying positive definite kernels on (the objects of a) category. By this I mean for a category $\mathcal{C}$, find a function $$ K: Ob\mathcal{C} \times Ob\...
11
votes
2answers
1k views

Physical interpretation of the Manifold Hypothesis

Motivation: Most dimensionality reduction algorithms assume that the input data are sampled from a manifold $\mathcal{M}$ whose intrinsic dimension $d$ is much smaller than the ambient dimension $D$. ...
0
votes
0answers
350 views

Artificial intelligence simulating mathematicians (what a distopia!)

This is kind of soft and naive question, so feel free to shame on me :) I start from the fact that, in my opinion, what humans are interested in about mathematics are things that we find deep and ...
30
votes
8answers
7k views

How useful is differential geometry and topology to deep learning?

After seeing this article https://www.quantamagazine.org/an-idea-from-physics-helps-ai-see-in-higher-dimensions-20200109/ I wanted to ask myself how useful of an endeavor would it be if one goes ...
15
votes
1answer
733 views

Are primes linearly separable?

Let $X_1,\cdots,X_n$ be finite subsets of some set $Z$. Then the symmetric difference metric space: $$d(X_i,X_j) = \sqrt{ |X_i|+|X_j|-2|X_i\cap X_j|}$$ can be embedded in Euclidean space. The value $|...