2
$\begingroup$

I am reading the proof of Theorem 1(a) in the paper that proposed the CLIME method for estimating precision matrix. I am puzzled by an inequality on Page 605 three lines above formula (29). I isolate the specific question as follows for your convenience.

Let $\mathrm{X} = (X_1,...,X_p)^T$ be a $p$-variate random vector. $\{\mathrm{X}_1,...,\mathrm{X}_n\}$ is an iid random sample from the distribution of $\mathrm{X}$. WOLG, assume $\mathbb{E}(\mathrm{X})=0$. Define $Y_{kij} = X_{ki}X_{kj} - \mathbb{E}(X_{ki}X_{kj})$,$ k\in\{1,...,n\}$,$ i\in\{1,...,p\}$,$ j\in\{1,...,p\}$. Suppose the following exponential-type tail condition holds: there exists some $0<\eta<1/4$ and a bounded constant $K$ such that $\log p/n\leq\eta$ and $$\mathbb{E} e^{\lambda X_i^2} \leq K<\infty, \quad \mbox{for all } |\lambda|\leq\eta, \mbox{ for all } i\in \{1,...,p\}.$$ Let $t=\eta\sqrt{\log p /n} $. Prove that $$ n t^2 \mathbb{E}\left(Y_{kij}^2 e^{t|Y_{kij}|} \right) \leq \eta^{-1}K^2\log p$$ holds.

$\endgroup$

1 Answer 1

2
$\begingroup$

The best I can prove has an extra $K$, but I think it does not matter too much for the proof of the original paper.

\begin{align} n t^2 \operatorname{\mathbb{E}}\left(Y_{kij}^2 e^{t|Y_{kij}|} \right) &= n (\eta\sqrt{\log p /n})^2 \operatorname{\mathbb{E}}\left(Y_{kij}^2 e^{\eta\sqrt{\log p /n}|Y_{kij}|} \right) \\ &= \eta^2\log p \operatorname{\mathbb{E}} \left(Y_{kij}^2 e^{\eta\sqrt{\log p /n}|Y_{kij}|} \right) \\ &= \log p \operatorname{\mathbb{E}}\left((\eta Y_{kij})^2 e^{\eta\sqrt{\log p /n}|Y_{kij}|} \right) \\ &\leq \log p \operatorname{\mathbb{E}}\left((\eta Y_{kij})^2 e^{\eta\cdot\eta^{1/2}|Y_{kij}|} \right) \\ &= \eta^{-1}\log p \operatorname{\mathbb{E}}\left((\eta^{3/2} Y_{kij})^2 e^{\eta^{3/2}|Y_{kij}|} \right)\\ &\leq \eta^{-1}K^3\log p \tag{1}\label{eq6} \end{align}

The first inequality is by $\log p/n\leq\eta$. The second inequality is proved as follows:

It suffices to show that \begin{equation} \tag{2}\label{eq1} \operatorname{\mathbb{E}}\left((\lambda Y_{kij})^2 e^{\lambda|Y_{kij}|} \right)\leq K^, \quad \mbox{for all } |\lambda|\leq\eta/2. \end{equation} Then by $0<\eta<1/4$ we have $\eta^{3/2}<\eta/2$, hence the result is proved.

Now prove \eqref{eq1}. Since $s^2\leq e^s$ for $s>0$, we have \begin{equation} \tag{3}\label{eq2} \operatorname{\mathbb{E}}\left((\lambda Y_{kij})^2 e^{\lambda|Y_{kij}|} \right)\leq \operatorname{\mathbb{E}}\left(e^{2\lambda|Y_{kij}|} \right). \end{equation} By definition, we have $$Y_{kij} = \frac{1}{4}\left\{(X_{ki}+X_{kj})^2 - (X_{ki}-X_{kj})^2 - \operatorname{\mathbb{E}}\left[(X_{ki}+X_{kj})^2 - (X_{ki}-X_{kj})^2 \right]\right\}.$$ Then following \eqref{eq2} we have \begin{align} \operatorname{\mathbb{E}}\left(e^{2\lambda|Y_{kij}|} \right) &= \operatorname{\mathbb{E}}\left(e^{\frac{1}{2}\lambda\left\lvert\left\{(X_{ki}+X_{kj})^2 - (X_{ki}-X_{kj})^2 - \operatorname{\mathbb{E}}\left[(X_{ki}+X_{kj})^2 - (X_{ki}-X_{kj})^2 \right]\right\}\right\rvert} \right) \\ &\leq \operatorname{\mathbb{E}}\left(e^{\frac{1}{2}\lambda(X_{ki}+X_{kj})^2 + \frac{1}{2}\lambda(X_{ki}-X_{kj})^2 + \operatorname{\mathbb{E}}\left[(X_{ki}+X_{kj})^2 \right] + \operatorname{\mathbb{E}}\left[(X_{ki}-X_{kj})^2 \right] } \right) \\ &= \operatorname{\mathbb{E}}\left(e^{\lambda(X_{ki}^2+X_{kj}^2) + \operatorname{\mathbb{E}}(X_{ki}^2+X_{kj}^2) } \right) \\ &= e^{\lambda\operatorname{\mathbb{E}}(X_{ki}^2+X_{kj}^2)} \operatorname{\mathbb{E}}\left(e^{\lambda(X_{ki}^2+X_{kj}^2) } \right). \tag{4}\label{eq3} \end{align} By Hölder's inequality, $$\operatorname{\mathbb{E}}\left(e^{\lambda(X_{ki}^2+X_{kj}^2) } \right) \leq \operatorname{\mathbb{E}}^{1/2}\left(e^{2\lambda X_{ki}^2 } \right) \operatorname{\mathbb{E}}^{1/2}\left(e^{2\lambda X_{kj}^2 } \right). $$ By the exponential-type tail condition, we have $$\operatorname{\mathbb{E}} e^{2\lambda X_i^2} \leq K, \quad \mbox{for all } |\lambda|\leq\eta/2.$$ Hence, \begin{equation} \tag{5}\label{eq4} \operatorname{\mathbb{E}}\left(e^{\lambda(X_{ki}^2+X_{kj}^2) } \right) \leq K. \end{equation} By Jensen's inequality, \begin{equation} \tag{6}\label{eq5} e^{\lambda\operatorname{\mathbb{E}}(X_{ki}^2+X_{kj}^2)} = e^{\operatorname{\mathbb{E}}\lambda X_{ki}^2}e^{\operatorname{\mathbb{E}}\lambda X_{kj}^2} \leq \operatorname{\mathbb{E}}e^{\lambda X_{ki}^2} \operatorname{\mathbb{E}}e^{\lambda X_{kj}^2} \leq K^2 \end{equation} Combining \eqref{eq3}, \eqref{eq4} and \eqref{eq5}, \eqref{eq6} is proved.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .