# Questions tagged [machine-learning]

The machine-learning tag has no usage guidance.

108
questions

**0**

votes

**0**answers

65 views

### Compression bounds on the Copeland–Erdős constant

Motivation:
Given the set of prime numbers $\mathbb{P} \subset \mathbb{N}$, the Copeland–Erdős constant $\mathcal{C}$ is defined as [1]:
\begin{equation}
\mathcal{C} = \sum_{n=1}^\infty p_n \cdot 10^{-...

**0**

votes

**0**answers

19 views

### Loss function for matrix with fixed trace

I have a matrix $P \in \mathbb{R}^{n \times m}$ where $n \gg m$ and each row of $P$ has norm $1$.
It is not hard to see that the $P^T P$ has a fixed trace $n$.
I am try to find a loss function to push ...

**0**

votes

**0**answers

25 views

### Analytic formula for minimum possible error for functions in RKHS ball, on a simple classificaiton problem

Let $X=\mathcal S_{d-1}$ be the unit-sphere in $\mathbb R^d$, and let $K:X \times X \to \mathbb R^d$ be a Mercer kernel (e.g the Laplace kernel). Let $\mathcal H_K$ be the induced RKHS, and for $R \ge ...

**66**

votes

**9**answers

6k views

### What are possible applications of deep learning to research mathematics?

With no doubt everyone here has heard of deep learning, even if they don't know what it is or what it is good for. I myself am a former mathematician turned data scientist who is quite interested in ...

**0**

votes

**0**answers

72 views

### Binary Regression : Is this an open problem in Mathematics/Statistics?

Let $X$ be a random variable which takes values from $\Omega = (0,1)^m$ with a probability distribution $p(x)$. Assume $p$ is a BV function with non zero total variation and $p(x)>0\forall x\in\...

**-1**

votes

**0**answers

8 views

### Coordinate expansion or multiple regressions

I was wondering if anyone could offer some advice on the most productive direction to head in when seeking to fit a regresion on the below data which is most entirely centered on zero.
Currently ...

**13**

votes

**1**answer

1k views

### Various authors of the Bourbaki's books

As far as I understand, each chapter of the Bourbaki's collection was written by one (or two?) specific authors. The book itself was reviewed, corrected and after all approved by the whole Bourbaki ...

**3**

votes

**1**answer

167 views

### Ricci flow for manifold learning

I know that mean curvature and diffusion-type flows are common in manifold learning because of their smoothing effects. I haven't seen Ricci flow used as much. Given that Ricci and diffusion-type ...

**0**

votes

**0**answers

93 views

### Derivative of the function of random variable

Suppose we have a function $\phi(X)$ of random variable $X$. Suppose both of $\phi(X)$ and $X$ are random variables. If $\phi$ is differentiable, how to calculate the derivative of $\phi(X)$ w.r.t. $...

**1**

vote

**1**answer

54 views

### Bayes risk of binary classification problem with conditionally independet covariates

In the setting of this problem, $\eta(\vec{x})$ is $P(Y=1|\vec{X}=\vec{x})$, $Y \in {0,1}$, $X \in R^d$. Being the true probability know, the classification rule is simply $\eta(\vec{x})>0.5 \...

**0**

votes

**0**answers

47 views

### UCB1 for Multiarmed bandits: understanding the proof

Regarding Multiarmed Bandits, I am analysing algorithm UCB1 presented in this paper and specifically the proof of Theorem 1, concerning the regret of the algorithm. At page 243 there is a step I do ...

**4**

votes

**1**answer

167 views

### Iterating projections to random halfspaces

Consider the following process:
Start with a set $S = \mathbb R^n$. Repeat $L$ times: choose a random orthonormal basis $u_1, \ldots, u_n$, and consider the cone $C = \{ \sum \alpha_i u_i : \alpha_i \...

**3**

votes

**0**answers

148 views

### Games and the right mathematical framework for GANs

Generative Adversarial Networks were introduced in http://papers.nips.cc/paper/5423-generative-adversarial-nets and has more than 20000 citations.
It is an important topic within deep learning.
Are ...

**0**

votes

**0**answers

28 views

### Bounds on the regularized risk minimizer and risk minimizer?

Some of this is taken from a similar post here, but with modifications.
In supervised machine learning, we typically take a Risk Minimization (RM) point of view when formulating a problem. So, what we ...

**7**

votes

**2**answers

454 views

### Mathematics of GANs (generative adversarial networks)

Generative Adversarial Networks were introduced in http://papers.nips.cc/paper/5423-generative-adversarial-nets and has more than 20000 citations.
The paper introduced key paradigm changes which ...

**0**

votes

**1**answer

299 views

### In linear regression, we have 0 training error if data dimension is high, but are there similar results for other supervised learning problems?

I tried posting this question on Cross Validated (the stack exchange for statistics) but didn't get an answer, so posting here:
Let's consider a supervised learning problem where $\{(x_1,y_1) \dots (...

**0**

votes

**1**answer

76 views

### Finite VC dimension > the number of free parameters

I'm looking for an example of the following:
A hypothesis class $\mathcal{H}$ such that
$\forall h \in \mathcal{H}$, the number of free parameters of $h$ is equal to $n \in \mathbb{N}$ (where $n < ...

**2**

votes

**0**answers

42 views

### What are some beginner's references on algebraically structured (statistical) models, and their connection with group actions and Fourier transform?

I asked this question on Cross Validated a few days ago, but didn't really get a favorable response, so asking here to see if I get any.
I'm looking at the description of a short-term position in ...

**1**

vote

**0**answers

55 views

### Relation between minimizer of regularized risk & risk in statistical learning theory

In supervised machine learning, we typically take a Risk Minimization (RM) point of view when formulating a problem. So, what we typically solve for is the following:
$$ R^L(h) = \underset{h\in\...

**0**

votes

**1**answer

244 views

### How large sample $m$ is enough [closed]

I have a $D$ probability distribution over $X =R^d$, i have two samples $s_1$ and $s_2$ from $D$, each having size $m_1$, $m_2$, a unit ball centered at origin $B(0)$, defined by $B(0)=\{x \in R^2: \|...

**4**

votes

**0**answers

91 views

### Can we show equivalence of two distributions based on their statistics?

Let $p,q$ be two distributions on $\mathbb{R}^d$. Let $f:\mathbb{R}^d\times\mathbb{R}^d\rightarrow\mathbb{R}$. Under what conditions does $\mathbb{E}_{x\sim p}f(x,z)=\mathbb{E}_{x\sim q}f(x,z)\ \...

**0**

votes

**1**answer

87 views

### Using a poset or directed graph as input for a neural network [closed]

I'm not sure if this is the right community to post this in but I would appreciate any help. As the title states, I'm trying to train a neural network using some unconventional input. I'm wondering if ...

**1**

vote

**0**answers

28 views

### Restriction of Rademacher Complexity

Let $F\subseteq C([0,1]^n,\mathbb{R})$ be a finite family of functions, which is non-empty. Let $A,B$ be subseteq of $[0,1]^n$, again non-empty, and let $Rad(C)$ denote the Rademacher complexity of ...

**1**

vote

**0**answers

52 views

### Bayesian inference of stochastically evolving model parameters

I have a question related to self-calibration in radio interferometry, but I will try to phrase it as generic as possible. I have a set of data points, $D = \{ d_{0, t_0}, d_{1, t_0}, ..., d_{M, t_0}, ...

**0**

votes

**0**answers

38 views

### Relation between test and train error with gradient descent iterates

My question is about establishing an inequality between population error and expected training error (i.e, expected training error < population error) for a model trained with gradient descent on a ...

**0**

votes

**0**answers

23 views

### parametric model with parameters described as gaussian processes

Let's assume that I have some data $y_{t_i}$ (i = 0, 1, ..., N) and a model $\hat{y}(a(t), b(t))$, where the parameters of my model (a, b) evolve with time t in a stochastic manner. I am wondering if ...

**1**

vote

**1**answer

161 views

### References for “second order” random walk on graphs (used in “node2vec” paper)?

The "word2vec" family of methods provided a great breakthrough in natural language processing.
The methods assign to each word a vector in $R^{n}$, such that the "similar" words ...

**5**

votes

**0**answers

94 views

### Distance of two points in Grassmannian using Plücker coordinate

Let $G(q,D)$ be the Grassmannian of $q$-dimensional vector spaces in $\mathbb{R}^D$, where $q \le D$ are positive integers. In the paper, a distance between two points of $G(q,D)$ are defined as ...

**8**

votes

**4**answers

2k views

### How to learn a continuous function?

Let $\Omega \subset \mathbb{R}^m$ be an open subset bounded with a smooth boundary.
Problem : Given any bounded continuous function $f:\Omega\to\mathbb{R}$, can we learn it to a given accuracy $\...

**3**

votes

**1**answer

74 views

### Hermite polynomial after rotation

When we consider the $n$-dimensional standard normal distribution, the orthogonal basis is $\{H_S(x)\}_{S}$ where $H_S(x) = \prod_{k=1}^n H_{s_k}(x_k)$. Here $H_*(x)$ is the normalized probabilist's ...

**1**

vote

**1**answer

150 views

### Upper bounding VC dimension of an indicator function class

I would like to upper bound the VC dimension of the function class $ F$ defined as follows:
$$ F := \left\{ (x,t) \mapsto \mathbb{1} \left( c_Q\min_{q \in Q} {\|x-q \|}_1 - t > 0 \right) \; | \; Q ...

**52**

votes

**5**answers

7k views

### Why do bees create hexagonal cells ? (Mathematical reasons)

Question 0 Are there any mathematical phenomena which are related to the form of honeycomb cells?
Question 1 Maybe hexagonal lattices satisfy certain optimality condition(s) which are related to it? ...

**7**

votes

**1**answer

290 views

### Does the plane clustered to minimize sum distances^2 to clusters centers ( inertia / “K-means”) produce hexagonal clusters / hexagonal lattice?

"K-means" is the most simple and famous clustering algorithm, which has numerous applications.
For a given as an input number of clusters it segments set of points in R^n to that given number of ...

**2**

votes

**1**answer

75 views

### Uniform Lipschitz function approximation by shallow neural networks

Fix $d\in \mathbb{N}$. Let $F_1$ be the set of all 1-Lipschitz functions mapping $[0, 1]^d$ to $\mathbb{R}$.
For $\varphi: \mathbb{R} \rightarrow \mathbb{R}$ and $m \in \mathbb{N}$, let $N_\varphi^m$ ...

**0**

votes

**1**answer

97 views

### Independence in a sequential problem with observations getting added to buckets

Consider a sequence of random observations $(O(t))_{t\geq 1}$, with $O(t)=(D(t),J(t),Y(t))$. Denote $\mathcal{F}(t) := \sigma(O(1),\ldots,O(t))$, the filtration induced by the first $t$ observations.
...

**8**

votes

**1**answer

194 views

### Books to develop a unified view of statistics and information theory?

I hope to understand the connection between statistics and information theory in a deep philosophical sense.
I suppose the best place to start would be David MacKay's Information Theory, Inference, ...

**5**

votes

**1**answer

258 views

### Abstract mathematical concepts/tools appeared in machine learning research

I am interested in knowing about abstract mathematical concepts, tools or methods that have come up in theoretical machine learning. By "abstract" I mean something that is not immediately related to ...

**2**

votes

**0**answers

126 views

### Inequality on the Kullback-Leibler divergence

Let us define the arithmetic, geometric, and harmonic means of $x,y \in \mathbb{R}$ weighted by $\alpha =(\alpha_x,\alpha_y) \in [0,1]$, respectively as
\begin{equation}
a_\alpha(x,y) = \frac{\...

**0**

votes

**2**answers

169 views

### Statistical divergence

Does anyone know about a statistical divergence of this type?
\begin{equation}
\text{D}(P||Q) = \frac{1}{2} \left[\text{KL}(M||P) + \text{KL}(M||Q)\right]
\end{equation}
where $M = \frac{1}{2} [P+Q]$....

**1**

vote

**0**answers

84 views

### Comparison of concentrations of different $L^p$-norms of (sub) Gaussian distributions

It's well-known that the Euclidean $2$-norm of subgaussian random vectors concentrates in high dimensions, e.g. when $X \sim \mathcal{N}(0,I_n),$ (or in general $X$ is subgaussian with independent co-...

**0**

votes

**2**answers

165 views

### Help with a definition of a two-person game in a referenced paper

In the paper "Finding Mixed Nash Equilibria of Generative Adversarial Networks" the authors write in equation (1) on page 2:
Consider the classical formulation of a two-player game with
finitely ...

**1**

vote

**0**answers

37 views

### What is the number of iterations needed for the message passing algorithm to converge when applied to an acyclic factor graph?

I understand that the message passing algorithm (Belief Propagation algorithm), when applied to a factor graph consists in an exchange in messages between the factor nodes and the variable nodes, ...

**9**

votes

**4**answers

2k views

### mathematical physics without partial derivatives

Remark: All the answers so far have been very insightful and on point but after receiving public and private feedback from other mathematicians on the MathOverflow I decided to clarify a few notions ...

**2**

votes

**0**answers

55 views

### Variational forms of non-convex functions

I am trying to understand what kind of variational forms exist for non-convex functions. Alternatively, are there conjugate forms which attain strong duality? For a non-convex function $f$, I am ...

**1**

vote

**0**answers

29 views

### sequential learning reference request [closed]

I would like to find a book for master math student about the following topics.
I don't know the field and I don't want to be lost in details so if it could be an straight forward please.
I'm a ...

**7**

votes

**0**answers

99 views

### Positive definite kernels on categories

I'm wondering if there is any work on studying positive definite kernels on (the objects of a) category. By this I mean for a category $\mathcal{C}$, find a function
$$
K: Ob\mathcal{C} \times Ob\...

**11**

votes

**2**answers

1k views

### Physical interpretation of the Manifold Hypothesis

Motivation:
Most dimensionality reduction algorithms assume that the input data are sampled from a manifold $\mathcal{M}$ whose intrinsic dimension $d$ is much smaller than the ambient dimension $D$. ...

**0**

votes

**0**answers

350 views

### Artificial intelligence simulating mathematicians (what a distopia!)

This is kind of soft and naive question, so feel free to shame on me :)
I start from the fact that, in my opinion, what humans are interested in about mathematics are things that we find deep and ...

**30**

votes

**8**answers

7k views

### How useful is differential geometry and topology to deep learning?

After seeing this article https://www.quantamagazine.org/an-idea-from-physics-helps-ai-see-in-higher-dimensions-20200109/ I wanted to ask myself how useful of an endeavor would it be if one goes ...

**15**

votes

**1**answer

733 views

### Are primes linearly separable?

Let $X_1,\cdots,X_n$ be finite subsets of some set $Z$. Then the symmetric difference metric space:
$$d(X_i,X_j) = \sqrt{ |X_i|+|X_j|-2|X_i\cap X_j|}$$
can be embedded in Euclidean space. The value $|...