Skip to main content
added 220 characters in body
Source Link
Iosif Pinelis
  • 127.7k
  • 8
  • 107
  • 229

$\newcommand\ep{\epsilon}$ $\newcommand\ch{\operatorname{ch}}$ This answer is based on Theorem 3 and formula (2), which yield $$P(|S_m|\ge x) \le2\exp\Big(-tx+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{0} $$ where $x$ and $t$ are any nonnegative real numbers, $S_m:=X_1+\dots+X_m$, and $X_1,\dots,X_m$ are any independent zero-mean random vectors in any separable Hilbert space $H$ with norm $|\cdot|$. Inequality (0) is also a special case of Theorem 3.1 for martingales in $(2,D)$-smooth separable Banach spaces; note that a Hilbert space is $(2,1)$-smooth.

In our case, let $H=\mathbb R^n$ with $|\cdot|:=\|\cdot\|_2$ and let $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n),$$ where $Y_1,\dots,Y_m$ are iid random variables (r.v.'s) such that $P(Y_i=j)=p_j$ for $j\in[n]:=\{1,\dots,n\}$ and $p_1+\dots+p_n=1$. Then we can write
$$S_m=m(q-p),$$ where $p$ and $q$ are as in the OP. So, for all $\ep\in(0,1/2]$ and all real $t\ge0$, $$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big). \tag{1} $$ Next, almost surely (a.s.) $$|X_i|^2=\sum_{j=1}^n1_{Y_i=j}-2\sum_{j=1}^np_j1_{Y_i=j}+\sum_{j=1}^np_j^2 \le1-2\times0+\sum_{j=1}^np_j^2\le2 $$ and $$E|X_i|^2=1-\sum_{j=1}^np_j^2\le1.$$ Also, $r(u):=\frac{e^u-u-1}{u^2}$ is increasing in real $u$ (with $r(0):=1/2$), whence $$r(u)\le r(\sqrt 2/2)=:c=0.64\ldots$$ for $u\le\sqrt 2/2$. Letting now $t_*:=\frac\ep{2c}<\ep\le1/2$, we have $t_*|X_i|\le\sqrt2/2$ a.s. So, a.s. for all $i\in[m]$
$$e^{t_*|X_i|}-t_*|X_i|-1=r(t_*|X_i|)t_*^2|X_i|^2\le c t_*^2|X_i|^2. $$ Hence, by (1), $$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m c t_*^2E|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$ So, for any $\delta>0$, $$P(|q-p|<\ep)\ge1-\delta $$ if $$m\ge\frac{4c}{\ep^2}\,\ln\frac2\delta $$ and $\ep\in(0,1/2]$. (Replacing $1/2$ in the condition $\ep\in(0,1/2]$ by a small enough positive real number, we can replace $c$ by a constant factor however close to $1/2$.)

$\newcommand\ep{\epsilon}$ $\newcommand\ch{\operatorname{ch}}$ This answer is based on Theorem 3 and formula (2), which yield $$P(|S_m|\ge x) \le2\exp\Big(-tx+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{0} $$ where $x$ and $t$ are any nonnegative real numbers, $S_m:=X_1+\dots+X_m$, and $X_1,\dots,X_m$ are any independent zero-mean random vectors in any separable Hilbert space $H$ with norm $|\cdot|$.

In our case, let $H=\mathbb R^n$ with $|\cdot|:=\|\cdot\|_2$ and let $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n),$$ where $Y_1,\dots,Y_m$ are iid random variables (r.v.'s) such that $P(Y_i=j)=p_j$ for $j\in[n]:=\{1,\dots,n\}$ and $p_1+\dots+p_n=1$. Then we can write
$$S_m=m(q-p),$$ where $p$ and $q$ are as in the OP. So, for all $\ep\in(0,1/2]$ and all real $t\ge0$, $$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big). \tag{1} $$ Next, almost surely (a.s.) $$|X_i|^2=\sum_{j=1}^n1_{Y_i=j}-2\sum_{j=1}^np_j1_{Y_i=j}+\sum_{j=1}^np_j^2 \le1-2\times0+\sum_{j=1}^np_j^2\le2 $$ and $$E|X_i|^2=1-\sum_{j=1}^np_j^2\le1.$$ Also, $r(u):=\frac{e^u-u-1}{u^2}$ is increasing in real $u$ (with $r(0):=1/2$), whence $$r(u)\le r(\sqrt 2/2)=:c=0.64\ldots$$ for $u\le\sqrt 2/2$. Letting now $t_*:=\frac\ep{2c}<\ep\le1/2$, we have $t_*|X_i|\le\sqrt2/2$ a.s. So, a.s. for all $i\in[m]$
$$e^{t_*|X_i|}-t_*|X_i|-1=r(t_*|X_i|)t_*^2|X_i|^2\le c t_*^2|X_i|^2. $$ Hence, by (1), $$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m c t_*^2E|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$ So, for any $\delta>0$, $$P(|q-p|<\ep)\ge1-\delta $$ if $$m\ge\frac{4c}{\ep^2}\,\ln\frac2\delta $$ and $\ep\in(0,1/2]$. (Replacing $1/2$ in the condition $\ep\in(0,1/2]$ by a small enough positive real number, we can replace $c$ by a constant factor however close to $1/2$.)

$\newcommand\ep{\epsilon}$ $\newcommand\ch{\operatorname{ch}}$ This answer is based on Theorem 3 and formula (2), which yield $$P(|S_m|\ge x) \le2\exp\Big(-tx+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{0} $$ where $x$ and $t$ are any nonnegative real numbers, $S_m:=X_1+\dots+X_m$, and $X_1,\dots,X_m$ are any independent zero-mean random vectors in any separable Hilbert space $H$ with norm $|\cdot|$. Inequality (0) is also a special case of Theorem 3.1 for martingales in $(2,D)$-smooth separable Banach spaces; note that a Hilbert space is $(2,1)$-smooth.

In our case, let $H=\mathbb R^n$ with $|\cdot|:=\|\cdot\|_2$ and let $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n),$$ where $Y_1,\dots,Y_m$ are iid random variables (r.v.'s) such that $P(Y_i=j)=p_j$ for $j\in[n]:=\{1,\dots,n\}$ and $p_1+\dots+p_n=1$. Then we can write
$$S_m=m(q-p),$$ where $p$ and $q$ are as in the OP. So, for all $\ep\in(0,1/2]$ and all real $t\ge0$, $$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big). \tag{1} $$ Next, almost surely (a.s.) $$|X_i|^2=\sum_{j=1}^n1_{Y_i=j}-2\sum_{j=1}^np_j1_{Y_i=j}+\sum_{j=1}^np_j^2 \le1-2\times0+\sum_{j=1}^np_j^2\le2 $$ and $$E|X_i|^2=1-\sum_{j=1}^np_j^2\le1.$$ Also, $r(u):=\frac{e^u-u-1}{u^2}$ is increasing in real $u$ (with $r(0):=1/2$), whence $$r(u)\le r(\sqrt 2/2)=:c=0.64\ldots$$ for $u\le\sqrt 2/2$. Letting now $t_*:=\frac\ep{2c}<\ep\le1/2$, we have $t_*|X_i|\le\sqrt2/2$ a.s. So, a.s. for all $i\in[m]$
$$e^{t_*|X_i|}-t_*|X_i|-1=r(t_*|X_i|)t_*^2|X_i|^2\le c t_*^2|X_i|^2. $$ Hence, by (1), $$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m c t_*^2E|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$ So, for any $\delta>0$, $$P(|q-p|<\ep)\ge1-\delta $$ if $$m\ge\frac{4c}{\ep^2}\,\ln\frac2\delta $$ and $\ep\in(0,1/2]$. (Replacing $1/2$ in the condition $\ep\in(0,1/2]$ by a small enough positive real number, we can replace $c$ by a constant factor however close to $1/2$.)

added 410 characters in body
Source Link
Iosif Pinelis
  • 127.7k
  • 8
  • 107
  • 229

$\newcommand\ep{\epsilon}$ $\newcommand\ch{\operatorname{ch}}$ This answer is based on Theorem 3 and formula (2), which yield $$P(|S_m|\ge x) \le2\exp\Big(-tx+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{0} $$ where $x$ and $t$ are any nonnegative real numbers, $S_m:=X_1+\dots+X_m$, and $X_1,\dots,X_m$ are any independent zero-mean random vectors in any separable Hilbert space $H$ with norm $|\cdot|$.

In our case, let $H=\mathbb R^n$ with $|\cdot|:=\|\cdot\|_2$ and let $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n),$$ where $Y_1,\dots,Y_m$ are iid random variables (r.v.'s) such that $P(Y_i=j)=p_j$ for $j\in[n]:=\{1,\dots,n\}$ and $p_1+\dots+p_n=1$. Then we can write
$$S_m=m(q-p),$$ where $p$ and $q$ are as in the OP. So, for all $\ep\in(0,1/2]$ and all real $t\ge0$, $$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big). \tag{1} $$ Next, almost surely (a.s.) $$|X_i|^2=\sum_{j=1}^n1_{Y_i=j}-2\sum_{j=1}^np_j1_{Y_i=j}+\sum_{j=1}^np_j^2 \le1-2\times0+\sum_{j=1}^np_j^2\le2 $$ and $$E|X_i|^2=1-\sum_{j=1}^np_j^2\le1.$$ Also, $r(u):=\frac{e^u-u-1}{u^2}$ is increasing in real $u$ (with $r(0):=1/2$), whence $$r(u)\le r(\sqrt 2/2)=:c=0.64\ldots$$ for $u\le\sqrt 2/2$. Letting now $t_*:=\frac\ep{2c}<\ep\le1/2$, we have $t_*|X_i|\le\sqrt2/2$ a.s. So, a.s. for all $i\in[m]$
$$e^{t_*|X_i|}-t_*|X_i|-1=r(t_*|X_i|)t_*^2|X_i|^2\le c t_*^2|X_i|^2. $$ Hence, by (1), $$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m cE|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$$$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m c t_*^2E|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$ So, for any $\delta>0$, $$P(|q-p|<\ep)\ge1-\delta $$ if $$m\ge\frac{4c}{\ep^2}\,\ln\frac2\delta $$ and $\ep\in(0,1/2]$. (Replacing $1/2$ in the condition $\ep\in(0,1/2]$ by a small enough positive real number, we can replace $c$ by a constant factor however close to $1/2$.)

$\newcommand\ep{\epsilon}$ $\newcommand\ch{\operatorname{ch}}$ This answer is based on Theorem 3 and formula (2), which yield $$P(|S_m|\ge x) \le2\exp\Big(-tx+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{0} $$ where $x$ and $t$ are any nonnegative real numbers, $S_m:=X_1+\dots+X_m$, and $X_1,\dots,X_m$ are any independent zero-mean random vectors in any separable Hilbert space $H$ with norm $|\cdot|$.

In our case, let $H=\mathbb R^n$ with $|\cdot|:=\|\cdot\|_2$ and let $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n),$$ where $Y_1,\dots,Y_m$ are iid random variables (r.v.'s) such that $P(Y_i=j)=p_j$ for $j\in[n]:=\{1,\dots,n\}$ and $p_1+\dots+p_n=1$. Then we can write
$$S_m=m(q-p),$$ where $p$ and $q$ are as in the OP. So, for all $\ep\in(0,1/2]$ and all real $t\ge0$, $$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big). \tag{1} $$ Next, almost surely (a.s.) $$|X_i|^2=\sum_{j=1}^n1_{Y_i=j}-2\sum_{j=1}^np_j1_{Y_i=j}+\sum_{j=1}^np_j^2 \le1-2\times0+\sum_{j=1}^np_j^2\le2 $$ and $$E|X_i|^2=1-\sum_{j=1}^np_j^2\le1.$$ Also, $r(u):=\frac{e^u-u-1}{u^2}$ is increasing in real $u$ (with $r(0):=1/2$), whence $$r(u)\le r(\sqrt 2/2)=:c=0.64\ldots$$ for $u\le\sqrt 2/2$. Letting now $t_*:=\frac\ep{2c}<\ep\le1/2$, we have $t_*|X_i|\le\sqrt2/2$ a.s. So, by (1), $$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m cE|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$ So, for any $\delta>0$, $$P(|q-p|<\ep)\ge1-\delta $$ if $$m\ge\frac{4c}{\ep^2}\,\ln\frac2\delta $$ and $\ep\in(0,1/2]$. (Replacing $1/2$ in the condition $\ep\in(0,1/2]$ by a small enough positive real number, we can replace $c$ by a constant factor however close to $1/2$.)

$\newcommand\ep{\epsilon}$ $\newcommand\ch{\operatorname{ch}}$ This answer is based on Theorem 3 and formula (2), which yield $$P(|S_m|\ge x) \le2\exp\Big(-tx+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{0} $$ where $x$ and $t$ are any nonnegative real numbers, $S_m:=X_1+\dots+X_m$, and $X_1,\dots,X_m$ are any independent zero-mean random vectors in any separable Hilbert space $H$ with norm $|\cdot|$.

In our case, let $H=\mathbb R^n$ with $|\cdot|:=\|\cdot\|_2$ and let $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n),$$ where $Y_1,\dots,Y_m$ are iid random variables (r.v.'s) such that $P(Y_i=j)=p_j$ for $j\in[n]:=\{1,\dots,n\}$ and $p_1+\dots+p_n=1$. Then we can write
$$S_m=m(q-p),$$ where $p$ and $q$ are as in the OP. So, for all $\ep\in(0,1/2]$ and all real $t\ge0$, $$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big). \tag{1} $$ Next, almost surely (a.s.) $$|X_i|^2=\sum_{j=1}^n1_{Y_i=j}-2\sum_{j=1}^np_j1_{Y_i=j}+\sum_{j=1}^np_j^2 \le1-2\times0+\sum_{j=1}^np_j^2\le2 $$ and $$E|X_i|^2=1-\sum_{j=1}^np_j^2\le1.$$ Also, $r(u):=\frac{e^u-u-1}{u^2}$ is increasing in real $u$ (with $r(0):=1/2$), whence $$r(u)\le r(\sqrt 2/2)=:c=0.64\ldots$$ for $u\le\sqrt 2/2$. Letting now $t_*:=\frac\ep{2c}<\ep\le1/2$, we have $t_*|X_i|\le\sqrt2/2$ a.s. So, a.s. for all $i\in[m]$
$$e^{t_*|X_i|}-t_*|X_i|-1=r(t_*|X_i|)t_*^2|X_i|^2\le c t_*^2|X_i|^2. $$ Hence, by (1), $$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m c t_*^2E|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$ So, for any $\delta>0$, $$P(|q-p|<\ep)\ge1-\delta $$ if $$m\ge\frac{4c}{\ep^2}\,\ln\frac2\delta $$ and $\ep\in(0,1/2]$. (Replacing $1/2$ in the condition $\ep\in(0,1/2]$ by a small enough positive real number, we can replace $c$ by a constant factor however close to $1/2$.)

added 410 characters in body
Source Link
Iosif Pinelis
  • 127.7k
  • 8
  • 107
  • 229

$\newcommand\ep{\epsilon}$ Let$\newcommand\ch{\operatorname{ch}}$ This answer is based on Theorem 3 and formula (2), which yield $$P(|S_m|\ge x) \le2\exp\Big(-tx+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{0} $$ where $x$ and $t$ are any nonnegative real numbers, $S_m:=X_1+\dots+X_m$, and $X_1,\dots,X_m$ are any independent zero-mean random vectors in any separable Hilbert space $H$ with norm $|\cdot|$.

In our case, let $H=\mathbb R^n$ with $|\cdot|:=\|\cdot\|_2$ and let $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n),$$ where $Y_1,\dots,Y_m$ beare iid random variables (r.v.'s) such that $P(Y_i=j)=p_j$ for $j\in[n]:=\{1,\dots,n\}$, where and $p_1+\dots+p_n=1$. LetThen we can write
$$S_m:=X_1+\dots+X_m=m(q-p),$$$$S_m=m(q-p),$$ where $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n)$$ and $p$ and $q$ are as in the OP. So, by formula (2), for all $\ep\in(0,1/2]$ and all real $t\ge0$, $$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{1} $$$$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big). \tag{1} $$ where $|\cdot|:=\|\cdot\|_2$. NextNext, almost surely (a.s.) $$|X_i|^2=\sum_{j=1}^n1_{Y_i=j}-2\sum_{j=1}^np_j1_{Y_i=j}+\sum_{j=1}^np_j^2 \le1-2\times0+\sum_{j=1}^np_j^2\le2 $$ and $$E|X_i|^2=1-\sum_{j=1}^np_j^2\le1.$$ Also, $r(u):=\frac{e^u-u-1}{u^2}$ is increasing in real $u$ (with $r(0):=1/2$), whence $$r(u)\le r(\sqrt 2/2)=:c=0.64\ldots$$ for $u\le\sqrt 2/2$. Letting now $t_*:=\frac\ep{2c}<\ep\le1/2$, we have $t_*|X_i|\le\sqrt2/2$ a.s. So, by (1), $$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m cE|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$ So, for any $\delta>0$, $$P(|q-p|<\ep)\ge1-\delta $$ if $$m\ge\frac{4c}{\ep^2}\,\ln\frac2\delta $$ and $\ep\in(0,1/2]$. (Replacing $1/2$ in the condition $\ep\in(0,1/2]$ by a small enough positive real number, we can replace $c$ by a constant factor however close to $1/2$.)

$\newcommand\ep{\epsilon}$ Let $Y_1,\dots,Y_m$ be iid random variables (r.v.'s) such that $P(Y_i=j)=p_j$ for $j\in[n]:=\{1,\dots,n\}$, where $p_1+\dots+p_n=1$. Let $$S_m:=X_1+\dots+X_m=m(q-p),$$ where $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n)$$ and $p$ and $q$ are as in the OP. So, by formula (2), for all $\ep\in(0,1/2]$ and all real $t\ge0$, $$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{1} $$ where $|\cdot|:=\|\cdot\|_2$. Next, almost surely (a.s.) $$|X_i|^2=\sum_{j=1}^n1_{Y_i=j}-2\sum_{j=1}^np_j1_{Y_i=j}+\sum_{j=1}^np_j^2 \le1-2\times0+\sum_{j=1}^np_j^2\le2 $$ and $$E|X_i|^2=1-\sum_{j=1}^np_j^2\le1.$$ Also, $r(u):=\frac{e^u-u-1}{u^2}$ is increasing in real $u$ (with $r(0):=1/2$), whence $$r(u)\le r(\sqrt 2/2)=:c=0.64\ldots$$ for $u\le\sqrt 2/2$. Letting now $t_*:=\frac\ep{2c}<\ep\le1/2$, we have $t_*|X_i|\le\sqrt2/2$ a.s. So, by (1), $$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m cE|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$ So, for any $\delta>0$, $$P(|q-p|<\ep)\ge1-\delta $$ if $$m\ge\frac{4c}{\ep^2}\,\ln\frac2\delta $$ and $\ep\in(0,1/2]$. (Replacing $1/2$ in the condition $\ep\in(0,1/2]$ by a small enough positive real number, we can replace $c$ by a constant factor however close to $1/2$.)

$\newcommand\ep{\epsilon}$ $\newcommand\ch{\operatorname{ch}}$ This answer is based on Theorem 3 and formula (2), which yield $$P(|S_m|\ge x) \le2\exp\Big(-tx+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big), \tag{0} $$ where $x$ and $t$ are any nonnegative real numbers, $S_m:=X_1+\dots+X_m$, and $X_1,\dots,X_m$ are any independent zero-mean random vectors in any separable Hilbert space $H$ with norm $|\cdot|$.

In our case, let $H=\mathbb R^n$ with $|\cdot|:=\|\cdot\|_2$ and let $$X_i:=(1_{Y_i=1}-p_1,\dots,1_{Y_i=n}-p_n),$$ where $Y_1,\dots,Y_m$ are iid random variables (r.v.'s) such that $P(Y_i=j)=p_j$ for $j\in[n]:=\{1,\dots,n\}$ and $p_1+\dots+p_n=1$. Then we can write
$$S_m=m(q-p),$$ where $p$ and $q$ are as in the OP. So, for all $\ep\in(0,1/2]$ and all real $t\ge0$, $$P(|q-p|\ge\ep)=P(|S_m|\ge m\ep) \\ \le2\exp\Big(-tm\ep+\sum_{i=1}^m E(e^{t|X_i|}-t|X_i|-1)\Big). \tag{1} $$ Next, almost surely (a.s.) $$|X_i|^2=\sum_{j=1}^n1_{Y_i=j}-2\sum_{j=1}^np_j1_{Y_i=j}+\sum_{j=1}^np_j^2 \le1-2\times0+\sum_{j=1}^np_j^2\le2 $$ and $$E|X_i|^2=1-\sum_{j=1}^np_j^2\le1.$$ Also, $r(u):=\frac{e^u-u-1}{u^2}$ is increasing in real $u$ (with $r(0):=1/2$), whence $$r(u)\le r(\sqrt 2/2)=:c=0.64\ldots$$ for $u\le\sqrt 2/2$. Letting now $t_*:=\frac\ep{2c}<\ep\le1/2$, we have $t_*|X_i|\le\sqrt2/2$ a.s. So, by (1), $$P(|q-p|\ge\ep)\le2\exp\Big(-t_*m\ep+\sum_{i=1}^m cE|X_i|^2\Big) \\ \le2\exp(-t_*m\ep+mct_*^2) =2\exp\Big(-\frac{m\ep^2}{4c}\Big). $$ So, for any $\delta>0$, $$P(|q-p|<\ep)\ge1-\delta $$ if $$m\ge\frac{4c}{\ep^2}\,\ln\frac2\delta $$ and $\ep\in(0,1/2]$. (Replacing $1/2$ in the condition $\ep\in(0,1/2]$ by a small enough positive real number, we can replace $c$ by a constant factor however close to $1/2$.)

added 177 characters in body
Source Link
Iosif Pinelis
  • 127.7k
  • 8
  • 107
  • 229
Loading
Source Link
Iosif Pinelis
  • 127.7k
  • 8
  • 107
  • 229
Loading