Skip to main content

All Questions

Filter by
Sorted by
Tagged with
1 vote
1 answer
160 views

Given iid $w_1,\dotsc,w_N \sim N(0,1/d)$ iid, find a simple matrix $A$ s.t $\|aa^T-A\|_\text{op}\to0$, where $a_i := E_{G \sim N(0,1)}[f(\|w_i\| G)]$

Let $d$ and $N$ be two large comparable integers, for example assume $$ N,d \to \infty, \quad d/N \to \gamma \in (0,\infty). $$ Let $w_1,\dotsc,w_N$ be iid from $N(0,(1/d)I_d)$ and let $f:\mathbb R \...
2 votes
1 answer
328 views

Matrix Bernstein's inequality: from tail probability to expectation

Let $X_i$ be independent, mean zero, $n\times n$, symmetric random matrices. $\|X_i\|\leq K$ almost sure for $\forall I$. We have matrix Bernstein's inequality for the tail probability as follows $$\...
10 votes
2 answers
1k views

Simple proof of sharp constant in DKW inequality

The DKW inequality says that if $F_n$ is the empirical CDF corresponding to real-valued random variables $X_1, \dots, X_n$ distributed identically and independently from a distribution with CDF $F$, ...
3 votes
0 answers
130 views

A Talagrand inequality for the supremum of partial sums over function classes under dependence. (Reference request)

As a consequence to the Talagrand concentration inequality, it is well known that for a measurable space $(S,\mathcal{S})$ and an i.i.d. sample $X_1,...,X_n$ of $S$-valued random variables, if $\...
2 votes
1 answer
415 views

High-probability lower bound for norm of least squares solution when both design matrix $X$ and response vector $y$ are random (and independent)

Let $n,d \to \infty$ with $n/d \to \gamma \in (0,\infty)$. Let $X$ be a random $n \times d$ matrix independent rows uniformly distributed on the the unit-sphere in $\mathbb R^d$ and let $y$ be a ...
3 votes
0 answers
92 views

Tighter Freedman's inequality for a special martingale difference sequence

Let $X_{1}, \ldots, X_{T} \in \{0, 1\}$ be a sequence of Boolean random variables with $$ \mathbb{E}[X_{t} | X_{1}, \dots, X_{t - 1}] = p_{t}. $$ Consider the sequence $Y_{t} := X_{t} - p_{t}$ (which ...
1 vote
0 answers
42 views

Sub-Gaussian analysis via bounded decomposition?

Let $\psi_\alpha(x) := \exp(x^\alpha)-1$. The Sub-Gaussian Norm $\lVert X \rVert_{\psi_2}$ of a random variable $X$ is defined as $$ \lVert X\rVert_{\psi_2} = \inf\{c>0\mid \mathbb{E}[\varphi_2(|X|/...
3 votes
1 answer
158 views

Sub-Gaussian concentration without the sub-Gaussian norm

A random variable $X$ is said to have sub-Gaussian tails with parameter $\sigma>0$ if $$\Pr[|X|\geq t] \leq 2\exp(-t^2/(2\sigma^2))$$ I am interested if $X_0, X_1$ are independent, and have sub-...
2 votes
3 answers
184 views

Existence and sharpness of Bernstein-type bounds on the moment-generating function

Let $X$ be a centred random variable with variance $\sigma^2$, and whose moment-generating function exists in an open neighbourhood of the origin. Say that $X$ satisfies a 'Bernstein-type' MGF bound ...
1 vote
1 answer
415 views

Approximate the singular values of a certain random dot-product kernel matrix (in the sense of El Karoui, Cheng-Singer, etc.)

Let $g:\mathbb R \to \mathbb R $ be a continuous function which is "sufficiently smooth" (e.g $\mathcal C^3$) around $0$, and "sufficiently integrable" (e.g integrable w.r.t $N(0,...
0 votes
1 answer
108 views

RMT for modified Wishard matrix $Y'Y$ (where $i$th row of $Y$ is zero if $|x_i^\top u| \le \theta$; else it equals $x_i$)

Let $n$ and $d$ be positive integers tending to infinity such that $d/n \to \phi \in (0,\infty)$. Let $X$ be an $n \times d$ random matrix with iid rows $x_1,\ldots,x_n$ from $N(0, \Sigma)$, where $\...
0 votes
0 answers
44 views

Large Deviation Principle for an adaptive sampling rule for Multi Armed Bandits

Consider the following adaptive strategy for sampling from a Multi Armed Bandit with $K$ arms: Split the $T$ rounds into $N (\in \mathbb{N})$ disjoint intervals. Each interval is indexed by $i=1,2,\...
5 votes
1 answer
1k views

Explicit constant for Carbery–Wright inequality

The Carbery–Wright inequality is a seminal result about the anti-concentration of polynomials of Gaussian random variables. See e.g. Meka, Nguyen, and Vu - Anti-concentration for polynomials of ...
3 votes
1 answer
379 views

Concentration inequality for norm of solution to nonlinear least-squares problem

Define the piecewise-linear function $\psi(t):=\max(t,0)$ for all $t \in \mathbb R$. Let $d,n,k \to \infty$ at the same rate (i.e $n \asymp k \asymp d$). Let $y_1,\ldots,y_n \in \{-1,1\}$ uniformly ...
3 votes
1 answer
373 views

Concentration of very dependent Markov chains

Consider the following simple Markov chain $ X_1\to X_2\to\cdots\to X_n $ where each $X_i$ is $\{-1,1\}$-valued and $X_1\sim\mathrm{Unif}(\{-1,1\})$ (such that the chain is stationary). The flip ...
1 vote
1 answer
284 views

Rate of convergence to uniform distribution

Let $p=(p(1),\ldots,p(N))$ be a discrete distribution on $[N]:=\{1,2,\ldots,N\}$ with full support (i.e all the $p(i)$'s are strictly positive and sum to $1$). Let $i_1,i_2,\ldots,i_T$ be an iid ...
1 vote
1 answer
230 views

VC dimension of a certain derived class of binary functions

Let $X$ be a measurable space and let $P$ be a probability distribution on $X \times \{\pm 1\}$. Let $F$ be a function class on $X$, i.e., a collection of (measurable) functions from $X$ to $\mathbb R$...
1 vote
1 answer
153 views

Minimax estimation rate of sparse vector $w_\star$, w.r.t to mixed norm $\|\hat w_n-w_\star\| := \|\hat w_n - w_\star\|_2 + \|\hat w_n-w_\star\|_q$

Let $n,d,s$ be positive integers with $s \le d$, and let $B_0(d,s)$ be the set of all (real) $d$-dimensional vectors with at most $s$ nonzero components. Given an $n \times d$ matrix $X$ with rows $...
3 votes
1 answer
130 views

Does a DKW-type inequality hold for the empirical CDF of a random vector on the sphere?

I've begun to study concentration of measure because of its relevance to statistical mechanics. In recent decades concentration inequalities have played a role in elucidating foundational conceptual ...
2 votes
1 answer
83 views

Azuma-Hoeffding for one-side bounded super-martingale sequence

Suppose we have a real-valued super-martingale difference sequence $\{X_k\}$ w.r.t. some filtration $\mathcal{F_k}$, i.e., $X_k$ is $\mathcal{F_k}$-measurable, and $$ \mathbb{E}[X_k|\mathcal{F}_{k-1}] ...
1 vote
0 answers
34 views

Discrepancy between probability measures, tested against bounded functions of bounded variance

When studying some concentration inequalities, it became relevant to consider the following discrepancy between two probability measures $\pi$ and $\nu$ (treating $\sigma \in \left( 0, \frac{1}{2} \...
0 votes
1 answer
78 views

Uniform concentration bound (function-valued random variable / continuous stochastic process)

I'm trying to consider a probability space $\Omega$ and $f(x,\xi):\mathcal{X}\times\Omega\to\mathbb{R}$ (stochastic process over space? or function-valued random variable?), where $\mathcal{X}\subset\...
10 votes
3 answers
4k views

Extension of the Azuma-Hoeffding inequality (when the differences are bounded with large probability)

Let $(X_i)$ be a super-martingale and suppose their differences are bounded ''with high probability'', that is $$\mathbb{P}(\exists\,i=1,\dots,n\text{ s.t. }|X_i-X_{i-1}|>c_i) \,\leq\, \epsilon$$ ...
2 votes
1 answer
386 views

A maximal inequality

Let $\{X_i\}_{i\in\mathbb{N}}$ be i.i.d. symmetric random variables, with $-1\leq X_i\leq 1$, $\mathbb{E}(X_i) =0$, $\mathbb{E}(X_i^2) = 1$. We have that: $$ P\left(\bigcap_{k = 1}^{n}\frac{|\sum_{i = ...
9 votes
2 answers
616 views

construction of a random measure with a given mean

Let me first pose a trivial question. Given a Borel probability measure $\mu$ on the real line, is it possible to construct a purely atomic random measure $M$ whose mean is $\mu$? The answer is ...
-1 votes
1 answer
163 views

Is it true that if a random vector has independent coordinates each bounded by $1$ then $P[ \|X\| \leq \epsilon\sqrt{n}] \leq (C\epsilon)^{n}$?

I'm studying Vershynin's well-written book on "High Dimensional Probability" and the third chapter on concentration of random vectors. Exercise 3.1.7 from the book is the following. Let $X =...
2 votes
1 answer
136 views

Concentration bound for a increasingly weighted sum of bernoulli random variables

Given $x_1,x_2,\ldots,x_n$ i.i.d. bernoulli random variables with $P(x_i=1)=\frac1n$. Given a constant $c=1+\frac{1}{m}, m\geq n$. Is there an explicit theorem that can derive a concentration argument ...
1 vote
1 answer
216 views

Rademacher complexity of function class $(x,y) \mapsto 1[|yf(x)-\alpha| \ge \beta]$ in terms of $\alpha$, $\beta$, and Rademacher complexity of $F$

Let $X$ be a measurable space and let $P$ be a probability distribution on $X \times \{\pm 1\}$. Let $F$ be a function class on $X$, i.e., a collection of (measurable) functions from $X$ to $\mathbb R$...
2 votes
1 answer
119 views

Simultaneous Concentration of $\sum_{i = 1}^{n} X_i^2$ and $\sum_{i = 1}^{n} X_i$ with $X_i$ iid. Poisson

Consider $n$ independent Poisson(1)-distributed random variables $(X_i)_{1 \leq i \leq n}$. This is a (hopefully more interesting) follow-up question to Super-exponential concentration for $\frac{\...
2 votes
0 answers
94 views

Concentration inequalities for functions of random binary strings

Let $(X_1,\ldots,X_n)$ be a vector in $\{0,1\}^n$ drawn uniformly at random among all vectors with exactly $k$ $1'$s. I am interested in inequalities for tail probabilities for the random variables $X,...
3 votes
1 answer
156 views

Concentration of measure on spheres with respect to a unitary of trace approximately zero

Cross-posted from MSE, where it hasn’t received any answer yet: This question arose out of my attempt to understand how a unitary of trace approximately zero acts on the unit sphere of a $n$-...
2 votes
0 answers
84 views

Concentration result for self-normalized empirical process

In Theorem 1.1 of this paper by Bercu, Gassiat and Rio, a concentration result is derived for the 'self-normalized' empirical process. Specifically, suppose that $(X,X_n)_{n \ge 1}$ is a sequence of i....
5 votes
2 answers
397 views

Why sum of samples without replacement is more concentrated than with replacement?

Set $n\le N$. Suppose $x_1,...,x_n$ are uniformly random variables taking value in $[N]$ In addition, let ${y_1,...,y_n}$ be an $n-$subset of $[N]$ that is chosen uniformly random among all $N \choose ...
1 vote
1 answer
115 views

How does Chernoff-Hoeffding bound with limited independence reduce to the usual generic CH bound with complete independence

As the title might suggest, I am referring to this paper https://www.cs.umd.edu/~srin/PDF/ch-bounds.pdf , titled : Chernoff-Hoeffding Bounds for Application with Limited Independence. The theorem in ...
0 votes
0 answers
116 views

Concentration bounds for sum of weighted sampling without replacement

Let $X$ be a collection of $2l$ non-negative numbers $X_1,X_2,\ldots,X_{2l}$. We draw $l$ weighted (proportional to values) samples without replacement from $X$. Let $S$ denote this set of $l$ samples....
0 votes
1 answer
231 views

Concentration inequalities for random sampling without replacement

Let a population $C$ consist of $N$ values $c_1, c_2, \cdots, c_N$, with $c_i\in \{0,1\}$. Let $X_1, X_2, \cdots, X_n$ denote a random sample without replacement from $C$ and let $Y_1, Y_2, \cdots, ...
1 vote
2 answers
221 views

Beating the $1/\sqrt n$ rate of uniform-convergence over a linear function class

Let $P$ be a probability distribution on $\mathbb R^d \times \mathbb R$, and let $(x_1,y_1), \ldots, (x_n,y_n)$ be an iid sample of size $n$ from $P$. Fix $\epsilon,t\gt 0$. For any unit-vector $w \in ...
4 votes
1 answer
189 views

Sign of error in the central limit theorem

Let $X_n$ and $Y_n$ be independent copies of two random variables $X$ and $Y$ with domain $\{-1,0,1\}$ for $n\in \mathbb{N}$. For a given $k\in \mathbb{N}$, I would like to find conditions on $X$ and $...
3 votes
2 answers
442 views

What happens to the Gaussian volume of a Borel set when it is translated?

Let $\gamma_n$ be the standard Gaussian measure on $\mathbb R^n$, $A \subseteq \mathbb R^n$ be Borel and $c \in \mathbb R^n$. Define the translate $A_c := c + A := \{c+a \mid a \in A\} = \{x \in \...
0 votes
2 answers
280 views

Bounds tighter than the additive Chernoff

Additive Chernoff Suppose $X_1, \ldots, X_n$ are i.i.d. random variables, taking values in $\{0,1\}$. Let $p=\mathrm{E}\left[X_i\right]$ and $\varepsilon>0$. \begin{gather*} \operatorname{Pr}\left(\...
1 vote
0 answers
57 views

Limiting value of expectation of trace of truncated Gram matrix

Let $n$ and $d$ be large positive integers such that $d/n = a \in (0,1)$, fixed. Let $x_1,\ldots,x_n$ be iid random vectors from $N(0,I_d)$. Fix $b \in (0,1]$ and a unit-vector $v \in \mathbb R^d$, ...
4 votes
1 answer
320 views

Sub-Gaussian random variables and convex ordering

Suppose that $X$ is a $1$-sub-Gaussian real-valued random variable, i.e. for all $t \in \mathbf{R}$, it holds that $\log \mathbf{E} \exp \left( t X \right) \leqslant \frac{1}{2} t^2 $. Does there ...
1 vote
1 answer
191 views

Concentration inequality for square roots

Given a sequence of (not-necessarily-iid) real-valued random variables $X_n$ that converge to $a\in\mathbb{R}$ in probability, suppose we have an exponential concentration inequality of the form $$ P(|...
1 vote
0 answers
131 views

Large-deviation inequalities for a class of simple random multivariate polynomials

Let $N$ be a large positive integer and let $[N] := \{1,2,\ldots,N\}$. For any $k$, let $K_{N,k}$ denote the collection of $k$-element subsets of $[N]$. Let $x=(x_1,\ldots,x_N)$ be a uniformly random ...
2 votes
1 answer
150 views

Normalized concentration inequality for empirical CDF (iid sum)

Consider the empirical and population CDF, $$ F_n(t) = \frac{1}{n} \sum_{i=1}^n 1\{X_i \leq t\} \quad \mbox{and} \quad F(t) = \mathbb{E} [F_n(t)], $$ where above $X_1, \dots, X_n$ are iid, real-...
1 vote
0 answers
121 views

Composing an Orlicz norm related to Bernstein's inequality?

This is related to my previous question, but is hopefully more precise. I would like to reason about tail-bounds for polynomial products of concentrated random variables in $R:=\mathbb{R}[x]/(x^n-1)$. ...
0 votes
1 answer
123 views

Is the product of sub-Gaussian polynomials in $\mathbb{R}[x]/(x^n-1)$ sub-Gaussian?

Let $\psi_\alpha(x) := \exp(x^\alpha)-1$. It is well-known that for $\alpha\geq 1$ that $$\lVert X\rVert_{\psi_\alpha} = \inf\{k>0\mid \mathbb{E}[\psi_\alpha(|X|/k)] \leq 1\}$$ defines an Orlicz ...
5 votes
1 answer
401 views

Lower tail of random rank one sums?

Let $\{x_i\}_{i\geq 1}$ be iid random elements of the sequence space $\ell^2(\mathbb{N})$; assume that $\|x_i\|_2 \leq 1$ almost surely. Let $\Sigma = \mathbb{E}[x_1 \otimes x_1]$. Define $$ \Sigma_n =...
3 votes
0 answers
70 views

Concentration for Hamming balls

It is well known that Lipschitz functions on the Boolean $n$-cube endowed with the Hamming metric satisfy concentration properties. Specifically, most of their values lie in a range of width $O(\sqrt ...
1 vote
1 answer
195 views

Concentration of a certain simple / well-structured random multilinear polynomial with growing degree

Let $k$ and $N_1$ be positive integers and set $N=kN_1$. Partition $[N] := \{1,2,\ldots,N\}$ $k$ disjoint from $G_1,\ldots,G_k$ of each of size $N_1$, and let $\mathcal T(k,N_1)$ be a transversal of ...

1
2 3 4 5
7