3
$\begingroup$

Consider $f\in L^2(I)$, where $I$ is the unit interval and $L^2$ is w.r.t. Lebesgue measure, and consider an approximation of $f$ denoted by $\tilde{f}\in L^2$.

The error in approximated the moments of $f$ by those of $\tilde{f}$ can be readily bounded by $\|f-\tilde{f}\|_2$, e.g., denoting the respective expectancies as $\mu$ and $\tilde{\mu}$, one has that

$$\left| \mu - \tilde{\mu} \right| \leq \|f-\tilde{f} \|_2 \, , $$

and if we define the variance in the usual way, as ${\rm Var}(f) = \mathbb{E}[(f-\mu)^2]$, then

$$\left| {\rm Var}(f) - {\rm Var}(\tilde{f}) \right| \leq \left(\mu +\tilde{\mu} + \|f\|_2 + \|\tilde{f}\|_2 \right) \|f-\tilde{f}\|_2 \, .$$

Questions:

  1. Are these bounds tight?
  2. Is there a known formula for the error in $\mathbb{E}[(f-\mu)^n]$ for all $n\in \mathbb{N}$?
  3. Is there a known textbook reference for these inequalities? It seems like I'm hardly the first to have discovered them.

Remark: The above bounds are obtained using mainly the triangle and Cauchy-Schwarz inequalities.

$\endgroup$

1 Answer 1

5
$\begingroup$

Let $X:=f$ and $Y:=\tilde f$. Let $\mu:=EX$ and $\nu:=EY$. Let $\|\cdot\|:=\|\cdot\|_2$. Let $D:=Y-X$. For any random variable $V$, let $\tilde V:=V-EV$.

Then the first displayed inequality can be rewritten as $ED\le\|D\|$. The upper bound here is attained if e.g. $D=1$. So, this bound is tight.

The second displayed inequality is in general false. E.g., suppose that $\mu=\nu<0$, $\tilde X=0$, and $P(\tilde Y=\pm1)=1/2$. Then the left-hand side of that inequality is $1$, whereas its right-hand side is $2\mu+|\mu|+\sqrt{\mu^2+1}<1$.

However, one can write \begin{multline*} \text{Var}\; Y-\text{Var}\; X=E(\tilde Y^2-\tilde X^2)=E\tilde D(\tilde X+\tilde Y) \\ \le\|\tilde D\|\,(\|\tilde X\|+\|\tilde Y\|) \le\|D\|\,(\|X\|+\|Y\|), \end{multline*} by the Cauchy-Schwarz and Minkowski inequalities. So, \begin{equation*} |\text{Var}\; Y-\text{Var}\; X| \le\|\tilde D\|\,(\|\tilde X\|+\|\tilde Y\|) \le\|D\|\,(\|X\|+\|Y\|). \tag{1} \end{equation*} The penultimate bound on $|\text{Var}\; Y-\text{Var}\; X|$ is attained e.g. when $Y=cX$ for a real constant $c>0$; if, in addition, $EX=0$, then the latter bound is attained as well. So, the latter two bounds are both tight, in their terms.

One can also adopt this approach to bound the "error" for the central moments $E(X-\mu)^n=E\tilde X^n$ of higher orders $n$ by writing $a^n-b^n=(a-b)\sum_{j=0}^{n-1} a^jb^{n-1-j}$ for $a=\tilde X$ and $b=\tilde Y$, and then using the H\"older and Minkowski inequalities, e.g., as follows: \begin{align*} |E\tilde X^n-E\tilde Y^n| &\le\|\tilde X-\tilde Y\|_n\;\sum_{j=0}^{n-1} \|\tilde X^j\tilde Y^{n-1-j}\|_{n/(n-1)} \\ &\le\|\tilde X-\tilde Y\|_n\;\sum_{j=0}^{n-1} \|\tilde X\|_n^j \|\tilde Y\|_n^{n-1-j} \tag{2} \\ &=\|\tilde X-\tilde Y\|_n\; \frac{\|\tilde X\|_n^n-\|\tilde Y\|_n^n}{\|\tilde X\|_n-\|\tilde Y\|_n}, \end{align*} the latter equality holding if $\|\tilde X\|_n\ne\|\tilde Y\|_n$. For the second inequality in the latter display, we use the obvious identity $\|f^m\|_p=\|f\|_{mp}^m$ for positive $m$ and $p$.

Clearly, the bound in (2) is a generalization of the first bound in (1).

For $p\ne2$, the inequality $\|\tilde X\|_p\le\|X\|_p$ will not hold in general. The optimal constant factor $c_p$ in the inequality $\|\tilde X\|_p\le c_p\|X\|_p$ for real $p>1$ was given in in Optimal Re-centering Bounds ... / arXiv. In particular, $c_3=\frac13\,(17+7\sqrt7)^{1/3}=1.0957\dots$.

$\endgroup$
10
  • $\begingroup$ Thanks! Two questions : (a) The second bound I've written is generally false - what limitations will make it generally right - that $EX$ and $EY$ are finite? $\endgroup$
    – Amir Sagiv
    Nov 4, 2017 at 10:05
  • $\begingroup$ (b) I tried to do it for $n=3$ and it became quite messy. I was hoping/wondering if there's a nice way to write it for general $n>0$. $\endgroup$
    – Amir Sagiv
    Nov 4, 2017 at 10:07
  • $\begingroup$ @AmirSagiv : Concerning your question (b): I have added details regarding central moments of higher orders $n$. $\endgroup$ Nov 5, 2017 at 0:50
  • 1
    $\begingroup$ @AmirSagiv : Concerning your question (a): Assuming that $EX$ and $EY$ are finite will not help, because this finiteness condition holds in the counterexample provided in my answer. $\endgroup$ Nov 5, 2017 at 1:08
  • $\begingroup$ are these bounds, for the mean and standard deviation, known before the ref. you just sent me? $\endgroup$
    – Amir Sagiv
    Jan 17, 2018 at 9:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.