1
$\begingroup$

Assume $\{h_j\}_{j\in \mathcal{N}}$ are independent Gamma random variables, each with potentially different distributions and parameters. I am looking for an upper bound for $\mathbb{E}\left[\max_{j \in \mathcal{N}} h_{j}\right]$ in closed form (not involving an expectation integral).

Here is a result that I received from ChatGPT:

$$ \mathbb{E}\left[\max_{j \in \mathcal{N}} h_{j}\right] \leq \max_{j \in \mathcal{N}} \mathbb{E}[h_{j}] + \frac{1}{2} \sqrt{\sum_{j \in \mathcal{N}} \text{Var}(h_{j})} $$

I verified this inequality through numerical simulations and found it to hold in various cases. However, I am unsure about its theoretical foundation.

Could you please:

  1. Verify this inequality theoretically?

  2. Provide references or derivations that lead to this result (or a similar one)?

  3. Suggest alternative upper bounds if this result is incorrect or suboptimal?

Any guidance or reference materials would be greatly appreciated!

$\endgroup$
6
  • 3
    $\begingroup$ I wouldn't trust ChatGPT on anything mathematical. $\endgroup$
    – gmvh
    Commented Nov 21 at 15:18
  • $\begingroup$ Yes, me too. So while the simulation results seem good, I have to find its critical proof. $\endgroup$
    – Lee White
    Commented Nov 21 at 15:21
  • $\begingroup$ You may check the following valid upper bound $$\inf_{0<t\le\min_i(\lambda_i)} \left \{ \frac{1}{t} \log \left ( \sum_{i=1}^n \left(\frac{\lambda_i}{\lambda_i-t}\right)^{\alpha_i} \right ) \right \}$$ where $\mu_i=\frac{\alpha_i}{\lambda_i}$ and $\sigma^2_i=\frac{\alpha_i}{\lambda^2_i}$ for $i\in[n].$ $\endgroup$
    – Amir
    Commented Nov 21 at 16:34
  • $\begingroup$ @Amir Thanks for your comment. Could you please share me the reference about the bound so that I can start from it. $\endgroup$
    – Lee White
    Commented Nov 21 at 17:26
  • $\begingroup$ I derived the bound myself. I can provide a proof if it preforms well. Could you check it using your simulation model and share the results? To find the inf, you can simply discretize the interval. $\endgroup$
    – Amir
    Commented Nov 21 at 17:36

3 Answers 3

3
$\begingroup$

$\newcommand\si\sigma$The inequality in question fails to hold if e.g. $\mathcal N=[n]:=\{1,\dots,n\}$, the $h_j$'s are i.i.d. exponential random variables, and $n=4$. This follows because then $$E\max_{j\in[n]} h_j=\int_0^\infty dx\,P(\max_{j\in[n]} h_j>x) =\int_0^\infty dx\,(1-(1-e^{-x})^n) =\sum_{k=1}^n\frac{(-1)^{k+1}}k\,\binom nk=H_n:=\sum_{k=1}^n\frac1k.$$


However, the inequality in question will hold if $\frac12$ there is replaced by $1$. Indeed, for any finite set $J$ and any independent family $(X_j)_{j\in J}$ of random variables with finite means $m_j:=EX_j$ and variances $\si_j^2$, let $$M:=\max_j X_j,\quad m:=\max_j m_j,\quad Y_j:=X_j-m_j. $$ Then $M\le m+\max_j Y_j$ and hence $$EM\le m+E\max_j Y_j.$$ Next, $$(E\max_j Y_j)^2\le E(\max_j Y_j)^2\le E\max_j Y_j^2 \le E\sum_j Y_j^2=\sum_j \si_j^2.$$ So, $$EM\le m+E\max_j Y_j\le m+|E\max_j Y_j|\le m+\sqrt{\sum_j \si_j^2}.\quad\Box$$

$\endgroup$
6
  • $\begingroup$ One should note that the bound (with the correct constant 1) will often be exceptionally loose. $\endgroup$
    – Alf
    Commented Nov 21 at 19:29
  • $\begingroup$ @Alf : The main loss is due to using the "union" inequality $E\max_j Y_j^2\le E\max_j Y_j^2$, which is, while quite commonly used, pretty loose when the tails of the $Y_j$'s are light enough. However, even when the $X_j$'s are i.i.d. exponential random variables (so that with rather light tails), even the inequality with the factor $\frac12$ will fail to hold for $n:=|J|=4,\dots,47$; so, in these cases, the bound with $1$ in place of $\frac12$ will be less than twice as large as $E\max_j X_j$. $\endgroup$ Commented Nov 21 at 22:01
  • $\begingroup$ Previous comment continued: Moreover, the bound with $1$ will then be less than three times as large as $E\max_j X_j$ for all $n=1,\dots, 328$. $\endgroup$ Commented Nov 21 at 22:01
  • $\begingroup$ The "union" inequality in my previous comment was of course meant to be $E\max_j Y_j^2\le E\sum_j Y_j^2$ . $\endgroup$ Commented Nov 21 at 22:57
  • $\begingroup$ @IosifPinelis Thanks for your answer. It is a nice Upper bound for the general mean max equation. $\endgroup$
    – Lee White
    Commented Nov 22 at 9:33
2
$\begingroup$

For $\{H_j\}_{j∈N}$ independent Gamma random variables as $H_j > 0$ then $max_{j\in N} H_j \leq \sum_{j\in N} H_j$. As $H$ are independent then $$\mathbb{E}[max_{j\in N} H_j] \leq \sum_{j\in N} \mathbb{E}[H_j]$$

$\endgroup$
2
  • $\begingroup$ Thank you for your answer. This upper bound is exactly right. But, it is too loose. $\endgroup$
    – Lee White
    Commented Nov 21 at 17:28
  • $\begingroup$ This argument doesn't require independence. The bound $\max_{j\in\mathcal{N}}h_j \leq \sum_{j\in\mathcal{N}} h_j$ implies $\mathbb{E}[\max_{j\in\mathcal{N}} h_j] \leq \mathbb{E}[\sum_{j\in\mathcal{N}} h_j] \stackrel{\ast}{=} \sum_{j\in\mathcal{N}}\mathbb{E}[h_j]$, where $\ast$ follows from linearity of expectation, which does not require independence. $\endgroup$ Commented Nov 22 at 2:27
2
$\begingroup$

For an alternative bound (which I just saw was referred to by Sandeep in the comments while revising this), you can adapt the standard argument for the expectation of the maximum of Gaussians. I do this below, where $M = \max_{i\in\mathcal{N}}h_j$. I will also assume that $h_j$ has shape $k_j>0$, scale $\theta_j>0$, and pdf $f_j(x) = \frac{1}{\Gamma(k_j)\theta_j^{k_j}}x^{k_j-1}\exp(x/\theta_j)$. One then gets that \begin{align*} \exp(t\mathbb{E}[M]) \leq\mathbb{E}[\exp [tM]]=\mathbb{E}[\max_{j\in\mathcal{N}} \exp(th_j)] \leq \sum_{j\in\mathcal{N}}\mathbb{E}[\exp(th_j)] \leq \sum_{j\in\mathcal{N}} (1-\theta_jt)^{-k_j}, \end{align*} valid for $t < \frac{1}{\max_{j\in\mathcal{N}}\theta_j}$. It follows that for such $t$ we get that

$$ \mathbb{E}[M] \leq \frac{1}{t}\ln(\sum_{j\in\mathcal{N}}(1-\theta_jt)^{-k_j}). $$ Noting that this is an instance of the LogSumExp function, and using the linked upper bound, we get that

$$\label{1}\tag{1} \forall t\in(0, (\max_{j\in\mathcal{N}}\theta_j)^{-1}): \mathbb{E}[M] \leq \frac{1}{t}\left(\max_{j\in\mathcal{N}}(k_j\ln\frac{1}{1-\theta_jt}) + \ln|\mathcal{N}|\right). $$ Under the approximation $$ \ln(\frac{1}{1-\theta_jt}) = \ln(\sum_{i = 0}^\infty (\theta_j t)^i) \stackrel{*}{\approx} \ln(1+\theta_j t) \leq \theta_jt, $$ the above bound simplifies to $$ \mathbb{E}[M] \leq \max_{j\in\mathcal{N}}\mathbb{E}[H_j] + \frac{\ln|\mathcal{N}|}{t}. $$ Choosing $t$ maximally, we get the bound $$ \mathbb{E}[M] \leq \max_{j\in\mathcal{N}}\mathbb{E}[h_j] + \max_{j\in\mathcal{N}}\theta_j\ln|\mathcal{N}|. $$

The approximation of $\ast$ is still not justified. It is of course formally false. That being said, it is easy to show that something close to it is true. For example, it is easy to see that $$ \sum_{i = 0}^\infty x^i = 1 + x + x^2\sum_{i = 0}^\infty x^i = 1 + x + \frac{x^2}{1-x} \leq 1 + x + \epsilon(1+x) = (1+\epsilon)(1+x) $$ provided $x^2 \leq \frac{\epsilon}{1+\epsilon}$, or in our setting $t \leq \frac{1}{\sqrt{1+\epsilon^{-1}}\max_{j\in\mathcal{N}}\theta_j}$. Under this choice of $t$, one gets the bound

$$\label{2}\tag{2} \mathbb{E}[M] \leq \max_{j\in\mathcal{N}}\mathbb{E}[h_j] + \max_{j\in\mathcal{N}}k_j\ln (1+\epsilon) + \sqrt{1+\epsilon^{-1}}\max_{j\in\mathcal{N}}\theta_j\ln|\mathcal{N}|, $$ where $\epsilon\in(0,1)$.

I wouldn't be surprised if there was a better bound one could use for $\ast$ than the one I used above, but hopefully Eq. \eqref{2} is better for your purposes than ChatGPT's answer. In particular, it has the benefit of scaling with $\ln|\mathcal{N}|$, rather than $O(\sqrt{|\mathcal{N}|})$, which may be preferable for $|\mathcal{N}|$ large.

Edit:

One can instead use the inequality

$$\tag{3}\label{3} \ln(1/(1-x)) \leq x + x^2, $$ valid for $x\in(0, 0.683803\dots)$. I do not have an exact value for the right endpoint, so will instead use that $2/3 \leq 0.683803\dots$ is relatively close, and Eq. \eqref{3} is valid for $x\in[0,2/3]$. If one chooses $\epsilon = 4/5$ such that the inequality $\ln(1/(1-x)) \leq (1+\epsilon)x$ is valid on the same interval $[0, 2/3]$, it is simple to see that the approximation $x+x^2$ is much tighter, see here.

Substituting this into Eq. \eqref{1}, we get that

$$ \mathbb{E}[M] \leq \max_{j\in\mathcal{N}} (\mathbb{E}[h_j] + t\mathsf{Var}[h_j]) + \frac{\ln|\mathcal{N}|}{t}, $$ for $t\in(0, \min(2/3, (\max_{j\in\mathcal{N}}\theta_j)^{-1})]$. So, if $2/3 \leq (\max_{j\in\mathcal{N}}\theta_j)^{-1}$, we may choose $t = 2/3$ to get the bound $$ \mathbb{E}[M] \leq \max_{j\in\mathcal{N}}(\mathbb{E}[h_j] + (2/3)\mathsf{Var}[h_j]) + \frac{3}{2}\ln|\mathcal{N}|. $$ If $2/3 > (\max_{j\in\mathcal{N}\theta_j})^{-1}$, we instead get that

$$ \mathbb{E}[M] \leq \max_{j\in\mathcal{N}}(\mathbb{E}[h_j] + \frac{\mathsf{Var}[h_j]}{\max_{i\in\mathcal{N}}\theta_i}) + \max_{i\in\mathcal{N}}\theta_i\ln|\mathcal{N}| \leq 2\max_{j\in\mathcal{N}}\mathbb{E}[h_j] + \max_{i\in\mathcal{N}}\theta_i\ln|\mathcal{N}|. $$

So, this should be better than the previously-derived bound. Perhaps there are even better variants of $\ast$ one can leverage, I don't know.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .