Skip to main content

All Questions

Filter by
Sorted by
Tagged with
6 votes
1 answer
203 views

Good upper-bound for $\mathbb E[|X-np|^r]$ where $X \sim \text{Binomial}(n,p)$ and $r \ge 1$

Disclaimer. Question moved from SE. Setup Let $X \sim \text{Binomial}(p, n)$, and $r \ge 1$. Question What is a good upper-bound for $\mathbb E[|X-np|^r]$ ? Solution for small $r$ If $r=2$, then ...
dohmatob's user avatar
  • 6,853
5 votes
0 answers
711 views

Concentration inequality for max component of a multivariate Gaussian in the general case

I am looking to bound the variance of the maximum component of a vector distributed multivariate Gaussian in the general case where the Gaussian distribution has arbitrary mean and full covariance ...
ted's user avatar
  • 283
1 vote
2 answers
250 views

Finite-sample deviation bound of empirical distribution from true distribution

Let $P=(p_1,\ldots,p_k) \in \Delta_k$ be distribution supported on set of size $k$ and let $\hat{P}_n$ be an empirical version of $P$ based on an iid sample of size $n$. Question What's a good non-...
dohmatob's user avatar
  • 6,853
4 votes
1 answer
239 views

Uniform inequality of the form $\text{Proba}(\sup_{v \in [-M,M]^k}|p^Tv-\hat{p}_n^Tv| \le \epsilon_n) \ge 1 - \delta$

Let $M > 0$, $k$ be a positive integer, and $\mathcal V:=[-M,M]^k$. Finally, let $p \in \Delta_k$, (where $\Delta_k$ is the $(k-1)$-dimensional probability simplex) and let $\hat{p}_n$ be an ...
dohmatob's user avatar
  • 6,853
1 vote
0 answers
123 views

Sanov-type finite-sample bound on $KL(P\|\hat{P}_n)$

Let $P$ be a distribution on an alphabet of size $k$ and let $\hat{P}_n$ be an empirical version of $P$ via $n$ i.i.d samples $a_1,\ldots,a_n \sim P$, i.e $\hat{P}_n := (1/n)\sum_{i=1}^n\delta_{a_i}$. ...
dohmatob's user avatar
  • 6,853
1 vote
1 answer
313 views

Bounds on difference between "logsumexp" and variance?

Let $Z$ be a random variable with finite moment-generating function $M_Z(\theta):=E[e^{\frac{1}{\theta}Z}]<\infty$ for all $\theta > 0$, and for $\delta \in (0,1]$, define $C_Z^\delta := \inf_{\...
dohmatob's user avatar
  • 6,853
3 votes
1 answer
826 views

concentration inequality for a weighted sum of independent but not identical binary variables

Let $\alpha\in[0,1]$ be a fixed constant, and let $w,x\in[0,1]^n$ be two vectors such that $\sum_i w_i x_i=\alpha$. Define $Y = \sum_i w_i X_i$, where $X_i \sim \operatorname{Bernoulli}(x_i)$, so it ...
guigux's user avatar
  • 617
2 votes
1 answer
847 views

Concentration inequality for quadratic form of Gaussian variables with non-idempotent matrix

Given $y \sim N(0,\sigma^2 I)$, and $M$ that is a symmetric matrix (not necessarily idempotent) what is the distribution of ${y^T M y}$? is there a high probability bound on $|{y^T M y}|$? Most ...
Enigman's user avatar
  • 123
1 vote
1 answer
365 views

Lower-bound probability of non-centered quadratic form

Let $X\sim N(\mu,\sigma^2I)\in \mathbb{R}^n$ be a non-centered ($\mu\neq 0$) Gaussian vector with independent coordinates. I'm wondering if there is any sharp lower bound of the following probability: ...
neverevernever's user avatar
1 vote
1 answer
137 views

Approximate $\log \mathbb E_P[\exp(th(x)]$ for a function $h$ which is lipschitz and has finite moments of order 1 and 2 w.r.t $P$

Let $P$ be a probability measure on a space $\mathcal X$ and $h: \mathcal X \rightarrow \mathbb R$ is measurable function with finite moments of order 1 and 2. I'm interested in approximating the ...
dohmatob's user avatar
  • 6,853
4 votes
1 answer
206 views

Inner product of sorted Gaussian vector

Suppose $X_1,\ldots,X_n$ are i.i.d. standard normal. I'm wondering how to analyze the following quantity: $$\left|\frac{X_{(1)}X_{(n)}+X_{(2)}X_{(n-1)}+\cdots+X_{(n)}X_{(1)}}{n}\right|$$ where $X_{(1)}...
neverevernever's user avatar
0 votes
1 answer
239 views

Anti-concentration: upper bound for $P(\sup_{a \in \mathbb S_{n-1}}\sum_{i=1}^na_i^2Z_i^2 \ge \epsilon)$

Let $\mathbb S_{n-1}$ be the unit sphere in $\mathbb R^n$ and $z_1,\ldots,z_n$ be a i.i.d sample from $\mathcal N(0, 1)$. Question Given $\epsilon > 0$ (may be assumed to be very small), what is ...
dohmatob's user avatar
  • 6,853
1 vote
1 answer
467 views

Sharp tail bounds for the maximum of an iid sample of a random variable supported on $[0, 1]$

Let $X_1,\ldots,X_n$ be an iid sample from a distribution supported on $[0, 1]$. Question What are some sharp concentration inequalities (i.e tail bounds) empirical statistic defined by $Z_n := \max(...
dohmatob's user avatar
  • 6,853
1 vote
1 answer
118 views

What is the order of the left tail of a mixture of non-central chi-square?

Let $\mu\sim N(0,1)$, $Z\sim N(\mu,1)$. Then $Z$ can be viewed as a mixture of Gaussians. It can also be viewed as a Gaussian but there is a prior for the mean. Let $X\sim\exp(\lambda)$ where the ...
neverevernever's user avatar
10 votes
2 answers
455 views

Largest deviations for uniform order statistics

Let $n >0$. Let $X_1,\ldots,X_n$ be i.i.d. uniform random variable on $[0,1].$ Denote by $X^{(1)}\leq X^{(2)} \leq \cdots \leq X^{(n)}$ their order statistics, and write $\Delta^{(i)} = \vert X^{(...
Gericault's user avatar
  • 245
3 votes
1 answer
294 views

Concentration inequalities specialized for log-likelihood / log-density functions

Let $P$ be a probability measure and $f$ be some probability density function (not necessarily related to $P$). Consider the function $$ L(X_1,\ldots,X_n) =\frac1n\sum_{i=1}^n\log f(X_i), \quad X_i\...
JohnA's user avatar
  • 710
4 votes
1 answer
1k views

Does variants of Bernstein and Freedman concentration inequalities exist with NO uniform bound on the range of RV or martingale differences

A classic formulation of the Bernstein inequality (from Wikipedia) is as follow: Let $X_1, \ldots, X_n$ be independent zero-mean random variables. Suppose that $|X_i|\leq M$ almost surely, for all $i$...
Jean Claude's user avatar
7 votes
1 answer
466 views

Martingale version of Bernstein-type inequality for (slightly) heavy-tailed distributions?

It is known that for sub-exponentially distributed martingale difference sequence, the following Bernstein-type inequality holds: $$ ℙ\left(\left| \sum_{i=1}^N a_i X_i \right| \ge t \right) \le 2\...
Nikolayevich's user avatar
3 votes
2 answers
731 views

Concentration inequality for sum of iid random variables that involve KL distance

Conider $X \in \mathbb{R}^d$ and $Y \in \{0,1\}$, and a joint distribution $p_{XY}(x,y)$, and a set of $N$ i.i.d. samples $\{(X_i,Y_i)\}_{i=1}^{N}$. Define $p_{X0} = p_{XY}(x,0)$ and $p_{X1} = p_{XY}(...
Jeff's user avatar
  • 482
3 votes
1 answer
196 views

Uniform Convergence for Vectors

$\textbf{Problem statement:}$ Let $\mathcal H:\mathcal X \rightarrow \{0,1\}$ be a class of Boolean functions for $\mathcal X \subset \mathbb R^n$, and let the VC Dimension of $\mathcal H$ be $VC_{...
AvidLearner's user avatar
1 vote
0 answers
376 views

Anti-concentration bounds for folded normal and inverse of gaussian variables

Are there any easy to use bounds on sums of the following kind : $$ \sum_{i = 1}^{i = N} |a_i| \geq P \\ a_i \sim \mathcal{N}(0, 1) \\ $$ and also for sums of the form : $$ \sum_{i = 1}^{i = M} \...
Govind Gopakumar's user avatar
1 vote
1 answer
249 views

On concentration of a sum random variable

Take a random variable defined as $$r=u_{11}v_{1}v_{1}+u_{12}v_{1}v_{2}+\dots+u_{n,n-1}v_{n}v_{n-1}+u_{nn}v_{n}v_{n}$$ where $v_{i}$ are independent uniform random variables from $\{0,\dots,b\}$, $u_{...
Turbo's user avatar
  • 13.9k
1 vote
1 answer
122 views

Variance bound of a functional

$X_1,\ldots,X_n$ are i.i.d standard normal random variables. $a_1,\ldots, a_n$ are constants with $a_i \in [\kappa_1, \kappa_2]$ for all $i$ and $\kappa_1>0$. $\hat c_n$ is given as the solution ...
Gourab Mukherjee's user avatar
5 votes
1 answer
372 views

What are some of results in low dimensional statistics that do not hold in high dimensions?

This question is partially inspired by the following MO post: What are some of the surprising results of finite sample statistical estimation? and current heated research front of high dimensional ...
Henry.L's user avatar
  • 8,071
3 votes
2 answers
319 views

Concentration inequality of joint event over time of a submartingale

Consider a discrete time submartingale $X_n$ with bounded difference $|X_n-X_{n-1}|\leq c$. With Azuma inequality we have the concentration of a single time event as $$ P(X_t-X_0 \leq -t) \leq exp\...
Sung-En Chiu's user avatar
4 votes
0 answers
141 views

Is there an example that both Berry-Essen bound and DKW bound are attained?

The Berry-Essen bound stated that $$\sup _{{x\in {\mathbb R}}}\left|\widehat{F_{n}(x)}-\Phi (x)\right|\leq C_{0}\cdot \psi _{0}$$ where $\psi _{0}(n)={\Big (}{\textstyle \sum \limits _{{i=1}}^{n}\...
Henry.L's user avatar
  • 8,071
2 votes
2 answers
1k views

Lower bound on number of samples for an epsilon delta approximation matching the Chernoff bound

So we have two biased coins, one comes out head w.p. $1/2+\epsilon$ and the other w.p. $1/2-\epsilon$. How many times should we flip these two coins to be able to tell them apart w.p. at least $\delta$...
shahrzad haddadan's user avatar
0 votes
0 answers
102 views

Probability of random variable being lesser than the other

Say there are two independent random variables, $X$ and $Y$, and we have samples $\{x_1,\dots x_n\},\{y_1,\dots y_n\}$. I am interested in bounding the probability of the event $C = \mathbb{1}_{X<Y}...
AvidLearner's user avatar
6 votes
0 answers
554 views

a variation on Hanson-Wright inequality

The classic Hanson-Wright inequality states that for a Gaussian random vector $\mathbf{x}\in\mathbb{R}^n$ distributed as $\mathcal{N}(\mathbf{0},\mathbf{I})$ and $\mathbf{A}\in\mathbb{R}^{n\times n}$ ...
mohi's user avatar
  • 859
4 votes
0 answers
76 views

How well does an estimator perform on another dataset?

Suppose $X \sim N(0, \Sigma)$ is a $d$-dimensional Gaussian random vector. And we have $2n$ $i.i.d$ sample $X_1, \ldots, X_{n}, \ldots, X_{2n}$. Let $\hat{\Sigma}_1 = \frac{1}{n}\sum_{i=1}^nX_i X_i^\...
Wuchen's user avatar
  • 515
10 votes
2 answers
847 views

Minimum separation among $m$ random points on an $n$-dimensional unit sphere

Consider $m$ points $v_1, \ldots, v_m \in R^{n}$, which are uniformly distributed on the $n$-dimensional unit sphere $S^{n-1} = \{v:\|v\|_2 = 1\}$. Let the minimum separation be $$ \rho = \min_{i,j\in{...
Minkov's user avatar
  • 1,127
4 votes
1 answer
347 views

Concentration of functional of Gaussian random variable

Suppose I have two Gaussian distributions $p(x) = \frac{1}{(2\pi)^{d/2}|\Sigma_p|^{1/2}}\exp(-\frac{1}{2}x^\top \Sigma_p^{-1} x)$ and $q(x) = \frac{1}{(2\pi)^{d/2}|\Sigma_q|^{1/2}}\exp(-\frac{1}{2}x^\...
Wuchen's user avatar
  • 515
7 votes
2 answers
605 views

Uniform Concentration Bounds on Weighted Sum of i.i.d. Bernoulli Random Variables

Let $\delta_1,...,\delta_n$ be $n$ independent identically distributed Bernoulli random variables with $\mathbb{P}(\delta_1=1)=p$. We consider a set $\Omega = \{\mathbf{a}:=(a_1,...,a_n)~|~a_i\in [0,c/...
tourzhao's user avatar
5 votes
2 answers
565 views

Concentration of U-statistics for exchangable distributions (and the unbounded case)

Consider the following so-called $U$-statistic of order 2: $$U = \frac1{\binom{m}{2}} \sum_{i < j} h(w_i,w_j)$$ where $w_1,\dots,w_m$ are IID from some distribution and $h$ is symmetric. If $|h(w_1,...
passerby51's user avatar
  • 1,731
10 votes
4 answers
645 views

Expected value of Bernoulli quadratic forms

Let $\mathbf{Y}\in\mathbb{R}^{n\times n}$ be a symmetric matrix. Let $\mathbf{x}\in\mathbb{R}^n$ be random vectors with entries i.i.d. $\pm 1$ with equal probability. I'm interested in a lower bound ...
Anahita's user avatar
  • 363
2 votes
0 answers
366 views

Convergence rate of Pearson correlation matrix

I am interested in (rather sharp if not the finest) tail/concentration bounds for the Pearson correlation matrix: let $X_1,\ldots,X_N \sim \mathcal{N}(0,1)$ be correlated random variables; let $\rho(...
mic's user avatar
  • 121
2 votes
0 answers
386 views

What is the concentration of measure for Gaussian random variables which are independent, but are transformed?

This might be a too easy question for Mathoverflow, but Googling led to similar questions and answers here (though not the one I was looking for). The question is split into two: I have a matrix $X \...
kloop's user avatar
  • 131
1 vote
0 answers
98 views

Small ball probabilities for functions of correlated normals

Let $f : \mathbb{R}^k \rightarrow \mathbb{R}$ and let $X$ be distributed k-dimensional normal with mean $0$ (with "arbitrary" covariance matrix). I am looking for references with bounds of the form: ...
rallen's user avatar
  • 111
4 votes
0 answers
1k views

Concentration of sum of independent random variables

Let $X_1, ..., X_n$ be i.i.d. sub-Gaussian random variables with mean $0$ and variance $1$. That is, we have $\Pr[|X_i| > t] \leq \exp(1-t^2/K^2)$ for all $t>0$ and a parameter $K$. Then we can ...
MCH's user avatar
  • 1,324
8 votes
2 answers
1k views

Does Multiplicative Version of Azuma's Inequality Hold?

It is known that there are multiplicative version concentration inequalities for sums of independent random variables. For example, the following multiplicative version Chernoff bound. Chernoff bound:...
Liwei Wang's user avatar
18 votes
1 answer
1k views

How fast can extreme eigenvalues of the average of random matrices converge to their expectation?

Suppose that $X_1,X_2,\ldots,X_m$ are independent $d\times d$ random matrices and let $\overline{X} := \frac{1}{m}\sum_{i=1}^m X_i$. One of the questions studied under the theory of random matrices is ...
sbahmani's user avatar
  • 181

1
2