1
$\begingroup$

$X$ is in the form of exponential family i.e. $$\mathbb{P_\theta}x = h(x)e^{\langle \theta,T(x)\rangle-\phi(\theta)}$$ where $\theta\in \mathbb{R}^d$. If $\nabla\phi(\theta)$ is L-Lipschitz i.e. $$\vert\nabla\phi(\theta_1) - \nabla\phi(\theta_2)\vert \leq L\vert\theta_1-\theta_2\vert,$$ how can we prove $Z = \langle v, T(X)\rangle$ for fixed $v$ satisfying $\Vert v\Vert_2=1$ is sub-Gaussian?

I thought $Z$ should be like the linear combination of $X$, but I failed to reach what the Lipschitz condition can control here.

$\endgroup$

1 Answer 1

1
$\begingroup$

$\newcommand\th\theta\newcommand\la\lambda\newcommand\R{\mathbb R}$Note that for $t\in\R^d$ we have
$$M_\th(t):=E_\th e^{t\cdot T(X)} =\int_{\R^d}dx\,h(x)e^{(t+\th)\cdot T(x)-\phi(\th)} =e^{\phi(t+\th)-\phi(\th)},$$ where $\cdot$ denotes the dot product. So, $$E_\th T(X)=\nabla M_\th(0)=\nabla\phi(\th)$$ and, for any real $\la$, $$E_\th e^{\la(Z-E_\th Z)} =E_\th\exp\{\la v\cdot T(X)-\la v\cdot E_\th T(X))\} \\ =\exp\{\phi(\la v+\th)-\phi(\th)-\la v\cdot \nabla\phi(\th)\}. \tag{1}\label{1}$$ By the mean-value theorem, for some $a\in(0,1)$ depending on $\la,v,\th$, we have $$\phi(\la v+\th)-\phi(\th)=\la v\cdot\nabla\phi(a\la v+\th).$$ Also, by the Lipschitz condition on $\nabla\phi$, $$|\nabla\phi(a\la v+\th)-\nabla\phi(\th)|\le La\la|v|\le L\la.$$ Thus, $$\phi(\la v+\th)-\phi(\th)-\la v\cdot \nabla\phi(\th) =\la v\cdot(\nabla\phi(a\la v+\th)-\nabla\phi(\th)) \le L\la^2$$ and hence, by \eqref{1}, $$E_\th e^{\la(Z-E_\th Z)} \le e^{L\la^2},$$ so that $Z$ is (uniformly) sub-Gaussian for all $\th$.

$\endgroup$
1
  • $\begingroup$ Thanks so much! $\endgroup$
    – dc3506
    Commented Apr 15, 2023 at 1:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.