3
$\begingroup$

Let $(g_i)_{i=1,...,d}$ sampled i.i.d. from a standard Gaussian, and $(\lambda_i)_{i=1,...,d}$ non-random s.t. $\max_i(\lambda_i)=1$ and $\lambda_i>0, \forall i$.

I am looking for the expectation of the Mahalanobis norm $E[\sqrt{\sum_{i=1}^d \lambda_i g_i^2}]$.

I know it in the special case when all $\lambda_i=1$, which is the expectation of a $\chi$-distribution with $d$ degrees of freedom. I also know that $\sqrt{\sum_{i=1}^d\lambda_i}$ is an upper bound by Jensen's inequality. But I would need a better estimate than that.

Update: The following lower bound would be good enough for what I need. Numerics indicate that it holds, now I'm trying to come up with a proof:

$E[\sqrt{\sum_{i=1}^d \lambda_i g_i^2}] \ge E[\sqrt{\sum_{i=1}^{T} u_i^2}]$, where $T=\lfloor \sum_{i=1}^d \lambda_i \rfloor$, and $(u_i)_{i=1,...,T}$ are i.i.d sampled from a standard Gaussian. If this holds then the LHS is known.

(My intuition on why it might hold is that the sum under the square root on the LHS has more independent terms than the one on the RHS, which makes it closer to its expectation than the other one is. Both have the same expectation and so a Jensen inequality would err less on the LHS term than on the RHS term. Not a clearly formed proof yet though.)

$\endgroup$
3
$\begingroup$

A greater and more general lower bound holds: $$(*)\qquad E f\Big(\sum_{i=1}^d \lambda_i g_i^2\Big) \ge E f(X_\lambda),$$ where $\lambda:=\lambda_1+\dots+\lambda_d$, $X_\lambda$ has the $\chi^2$ distribution with $\lambda$ degrees of freedom (that is, the $\text{Gamma}(\lambda/2,2)$ distribution), and $f$ is any concave function from $[0,\infty)$ to ${\mathbb{R}}$.

Indeed, by a standard induction argument (cf. e.g. the proof of Lemma 5.1.1 in normal domination), conditioning on all the $g_i$'s except one of them and using the convolution property $\text{Gamma}(\alpha_1,\beta)*\text{Gamma}(\alpha_2,\beta)=\text{Gamma}(\alpha_1+\alpha_2,\beta)$ for all positive real $\alpha_1,\alpha_2,\beta$, one sees that without loss of generality $d=1$.

By Corollary 5.7 (with $n=1$, $k=2$, and $I=[0,\infty)$) in dual cones or Corollary 5.8 therein (with $n=1$, $k=2$, and $I={\mathbb{R}}$), it suffices to prove $(*)$ for ($d=1$ and) $f=f_t$, where $t\in[0,\infty)$ and $f_t(x):=-(x-t)_+$ for all real $x$. That is, it suffices to prove that $$(**)\qquad G_1(t)\le F_1(t)$$
for all real $t\ge0$ and $\lambda\in(0,1)$, where $F_1(t):=F_{1,\lambda}(t):=\int_t^\infty(x-t)_+\,p(x)\,dx$, $G_1(t):=G_{1,\lambda}(t):=\int_t^\infty(x-t)_+\,q(x)\,dx$, and $p$ and $q$ are the $\text{Gamma}(\lambda/2,2)$ and $\text{Gamma}(1/2,2\lambda)$ density functions, respectively.

Introduce also $F(t):=-F_1'(t)=\int_t^\infty p(x)\,dx$ and $G(t):=-G_1'(t)=\int_t^\infty q(x)\,dx$. Clearly, $F(\infty-)=G(\infty-)=0$. Also, the ratio $q/p$ is incr.-decr. (on $I=[0,\infty)$) -- that is, for some $c\in I$ the ratio $q/p$ is increasing on $[0,c)$ and decreasing on $(c,\infty)$. So, by the l'Hospital-type rule for monotonicity given in Proposition 4.3 in l'Hospital-mono, the ratio $G/F$ is incr.-decr., so that for some $C\in I$ the ratio $G/F$ is increasing on $[0,C)$ and decreasing on $(C,\infty)$. In fact, $C>0$, because in a right neighborhood of $0$ one has $q<p$ and hence $G>F$, whereas $G(0)=F(0)=1$. By another application of the same l'Hospital-type rule for monotonicity, for some $C_1\in I$ the ratio $G_1/F_1$ is increasing on $[0,C_1)$ and decreasing on $(C_1,\infty)$. In fact, $C_1=0$, because $G_1(0)=F_1(0)=\lambda$ and in a right neighborhood of $0$ one has $G>F$ and hence $G_1<F_1$. That is, $G_1/F_1$ is decreasing on $[0,\infty)$, whence $G_1/F_1\le G_1(0)/F_1(0)=1$ on $[0,\infty)$, so that $(**)$ follows.

Addendum: Details on the reduction to the case $d=1$. As mentioned above, this reduction argument is quite similar to the argument in the proof of Lemma 5.1.1 in normal domination. Suppose that the inequality in $(*)$ holds for $d=1$. For $i=0,\dots,d$, introduce $$R_i:=\lambda_1 g_1^2+\dots+\lambda_i g_i^2+X_{\lambda_{i+1}}+\dots+X_{\lambda_d},$$ where the r.v.'s $g_1,\dots,g_d,X_{\lambda_1},\dots,X_{\lambda_d}$ are independent, with $X_{\lambda_i}$ having the $\text{Gamma}(\lambda_i/2,2)$ distribution for each $i=1,\dots,d$. Let $E_i$ denote the conditional expectation given $g_1,\dots,g_{i-1}, X_{\lambda_{i+1}},\dots,X_{\lambda_d}$. Note that
$R_i-\lambda_i g_i^2=\lambda_1 g_1^2+\dots+\lambda_{i-1} g_{i-1}^2+X_{\lambda_{i+1}}+\dots+X_{\lambda_d}$ is a function of $g_1,\dots,g_{i-1}, X_{\lambda_{i+1}},\dots,X_{\lambda_d}$. Since $f$ is concave, the (random) function $f_i$ given by the formula $f_i(x):=f(R_i-\lambda_i g_i^2+x)$ for all real $x$ is concave, for each $i$. So, by $(*)$ with $d=1$, for all $i=1,\dots,d$,
$$ E_i f(R_i) = E_i f_i(\lambda_i g_i^2) \ge E_i f_i(X_{\lambda_i}) = E_i f(R_{i-1}) $$ and hence $E f(R_i) \ge E f(R_{i-1})$. Thus,
$$E f\Big(\sum_{i=1}^d \lambda_i g_i^2\Big)=E f(R_d) \ge E f(R_0) = E f(X_\lambda),$$ so that $(*)$ follows in general. For the last equality displayed above, one uses the mentioned convolution property of the Gamma family of distributions.

$\endgroup$
  • $\begingroup$ Actually the first step is not clear to me, can you please clarify: How do we get a convolution with $\beta$ staying the same? I thought the $\lambda_i$'s change the $\beta$ parameter? $\endgroup$ – axk Apr 21 '15 at 4:26
  • 1
    $\begingroup$ axk: I have added requested details on the reduction to the case $d=1$. $\endgroup$ – Iosif Pinelis Apr 21 '15 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.