Search Results

If $X \sim Normal(0,1)$, then we have the tail bound: $$ (*) \qquad\Pr[X > t] \leq \mathcal{O}\left(\frac{e^{-t^2/2}}{t}\right) .$$ Now for general variables $X$, a nice condition is that $X$ be su …

A self-contained proof. Step 1: $\mathbb{E} \|\hat{P}_n - P\|_2^2 \leq \frac{1}{n}$. Step 2: McDiarmid's inequality. Let $X_i$ be the number of samples of $i \in \{1,\dots,k\}$. Then $X_i \sim \te …

The way this is usually put, the answer is that to achieve $\Pr[\|\hat{P}-P\|_1 \leq t] \geq 1- \delta$, one needs a sample size $N = \Theta\left(\frac{m + \ln(1/\delta)}{t^2}\right)$. The answer is t …

Here's a step that seems nice enough to point out. It still leaves a parameter to pick, and I'm not sure it's ever better than applying Bernstein, but it does something different. We can get a probab …

I'm looking for "tail-bound-like" inequalities that look like this (I state a specific setting but more general settings are interesting): Let $D$ be a distribution on a set of "nice" functions $g …

Adding to Iosif Pinelis' answer, there are two points here. First, as he says, the fact that we have a martingale rather than i.i.d. variables doesn't change much as proofs generally extend. So, secon …

If we let $P$ put probability $\frac{1}{2}$ each on $Y \in \{\pm 1\}$ independent of $X$, then for every classifier (from any class!) $$P(\text{sign}(h(x)) \neq y) = \frac{1}{2}$$ so $\text{err}_{\ma …