# Questions tagged [machine-learning]

The tag has no usage guidance.

77 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
308 views

### Profiles of very high dimensional functions

This question comes from trying to understand the recent success of deep neural nets. Neural networks just (crudely speaking) create a very complicated function of very many variables, and then ...
• 96.1k
165 views

### Worst margin when halving a hypercube with a hyperplane

Consider the $n$-cube $C_n=\lbrace-1,1\rbrace^n$ and the problem of partitioning it into halves with hyperplanes through the origin that avoid all its points. We can parameterize the hyperplanes by ...
• 1,065
326 views

### Monotonicity of log determinant of Gaussian kernel matrix

Let $$k({x},{y}) = \sigma \exp\left(-\frac{(x-y)^2}{2\theta^2}\right)$$ be a squared-exponential (Gaussian) kernel, with $\sigma,\vartheta>0$. Consider, for a set of $N$ ...
120 views

• 6,843
37 views

### Stochastic gradient descent in 'stronger' settings

I am minimzing a function $F(x) = \mathbb E(f(x,\Xi))$ where $\Xi$ is some random value, by a stochastic gradient descent that generates a random number $\xi$ from the distribution of $\Xi$ at each ...
• 686
80 views

### A question about fundamental invariants in the context of neural networks

I'm reading in depth the first part of the following paper: https://arxiv.org/pdf/1804.10306.pdf, paying specific attention to the following result, that I re-write here for the sake of convenience: [...
• 133
105 views

Let $\mathcal X$ be any set, and $f:\mathcal X\times\mathbb R^n\to\mathbb R$ be a differentiable model, meaning that for any fixed first argument, $f$ is differentiable in its second argument. Then we ...
• 633
49 views

### What are some beginner's references on algebraically structured (statistical) models, and their connection with group actions and Fourier transform?

I asked this question on Cross Validated a few days ago, but didn't really get a favorable response, so asking here to see if I get any. I'm looking at the description of a short-term position in ...
• 223
208 views

### Inequality on the Kullback-Leibler divergence

Let us define the arithmetic, geometric, and harmonic means of $x,y \in \mathbb{R}$ weighted by $\alpha =(\alpha_x,\alpha_y) \in [0,1]$, respectively as a_\alpha(x,y) = \frac{\...
• 245
91 views

### Variational forms of non-convex functions

I am trying to understand what kind of variational forms exist for non-convex functions. Alternatively, are there conjugate forms which attain strong duality? For a non-convex function $f$, I am ...
193 views

### Extension of universal approximation theorem

Let $I_d:=[0,1]^d$ with $d\ge 2$. Define $C(I_d):=\{F: I_d\to\mathbb R \mbox{ is continuous}\}$ and N(I_d):=\{F\in C(I_d): F(x)=\sum_{k=1}^n f_k(v_k\cdot x), \mbox{ where } n\ge 1 \mbox{ and } f_1,\...
1 vote
29 views

### Cluster minimizing sum of cost of clusters

Given a dataset $X,$ having $p$ features, organize the units $x_i \in X$ into fixed number of clusters $g,$ with fixed cluster size $B.$ Clustering policy: minimize the sum of a linear combination of ...
1 vote
40 views

### Constrained random sampling from partitioned sets with quotas

Let $D$ be a finite set, $\mathcal{P} = \{D_{i,j}\}_{(i,j) \in I \times J}$ a partition of $D$, $N: J \to \mathbb{N}$ a quota function, and $k \in \mathbb{N}^+$. A subset $F \subseteq D$ is considered ...
1 vote
23 views

### Symplectic aggregation over times steps

I am trying to achieve the following: given a sequence of phase space points $\left\{z_j\right\}=\left\{\left(q_j, p_j\right)\right\}$ for $j=1, \ldots, T$. Goal: Project this sequence to a single ...
• 29
1 vote
45 views

### Interpolation in convex hull

I'm reading a paper, Learning in High Dimension Always Amounts to Extrapolation, that provides a result I don't understand. It provides this theorem which I do understand: Theorem 1: (Bárány and ...
1 vote
30 views

### A network to transform/predict one probability distribution to another

I have a random variable of a particular density (e.g., normal), and a known probability distribution (e.g., mixture Gaussian). I used a simple KL measure to predict/transform one another. Now I need ...
1 vote
163 views

### Locally "unshortable" paths in graphs

Setup: Consider a connected graph G, with diameter "d". Informally: Trivially (by definition of diameter), taking any path $P$ any nodes $P(i) , P(i+k)$ for $k>d$ can be connected by a ...
• 24.5k
1 vote
80 views

### Approximation of continuous function by multilayer Relu neural network

For continuous/holder function $f$ defined on a compact set K， a fix $L$ and $m_1,m_2,\dots,m_L$, can we find a multilayer Relu fully connected network g with depth $L$ and each $i$-th layer has width ...
• 781
1 vote
128 views

### Matrix valued word embeddings for natural language processing

In natural language processing, an area of machine learning, one would like to represent words as objects that can easily be understood and manipulated using machine learning. A word embedding is a ...
• 28.4k
1 vote
Let $x_1,\ldots,x_n \in \mathbb R^d$ and $y_1,\ldots,y_n \in \{\pm 1\}$, and $\epsilon, h \gt 0$. Define $\theta(t) := Q((t-\epsilon)/h)$, where $Q(z) := \int_{z}^\infty \phi (z)\mathrm{d}z$ is the ...