Variance lower bound for natural exponential family

Question

Let $Q$ be a probability measure on $\mathbb{R}$. Let $$Q_h(dy) = e^{y \cdot h} Q(dy) / M(h) \quad \text{where} \quad M(h) = \int e^{y \cdot h} Q(dy)$$ defined for $h \in (-c,\infty)$ with some $c > 0$. Let $Y_h \sim Q_h$ and $V(h) := EY_h^2 - (EY_h)^2$. Then, does it follow that $$\exists c' > 0,\ h_0 > -c\ \text{ s.t. }\ \forall h \geq h_0,\ V(h) \geq e^{-c' h} \text{?}$$

Answer 1

The desired result obviously fails to hold if the probability measure $Q$ is degenerate, that is, supported on a singleton set. Indeed, then $V(h)=0$ for all $h$.

It is much harder to construct a counterexample with a non-degenerate $Q$. The idea of such a counterexample, given below, is to make the distribution $Q$ highly lacunary, in order to make the tilted distribution $Q_h$ very close to a degenerate one, for large $h$.

Indeed, let \begin{equation*} Q:=\sum_{i\ge1}p_i\delta_{2^i}, \end{equation*} where $\delta_a$ is the Dirac distribution supported on the set $\{a\}$, $p_i:=Ce^{-4^i}$, and $C$ is the normalizing constant. Then \begin{equation*} V(h)=\frac IJ, \tag{10}\label{10} \end{equation*} where \begin{equation*} I:=I(h):=\frac12\sum_{i,j\ge1}(2^j-2^i)^2z_i(h)z_j(h) \\ \le S:=S(h):=\sum_{1\le i<j}4^j z_i(h)z_j(h), \tag{20}\label{20} \end{equation*} \begin{equation*} J:=J(h):=M(h)^2=\Big(\sum_{i\ge1}z_i(h)\Big)^2, \tag{30}\label{30} \end{equation*}\begin{equation*} z_i(h):=e^{h2^i}p_i. \end{equation*}

For large natural $k$, let now \begin{equation*} h=h_k:=2^{k+1}, \tag{35}\label{35} \end{equation*} so that \begin{equation*} z_i(h_k)=y_i:=y_{k,i}:=C\exp(2^{k+1+i}-4^i). \end{equation*} Then
\begin{equation*} \frac{y_{i+1}}{y_i}=\exp(2^{k+i}(2-3\times2^{i-k})) \\ \le\exp(-2^{k+i})\le\exp(-2^{i+1}) \text{ for natural $i\ge k$} \tag{40}\label{40} \end{equation*} and \begin{equation*} \frac{y_{i+1}}{y_i}\ge\exp(2^{k-1+i})\ge1 \\ \text{ for natural $i<k$.} \tag{50}\label{50} \end{equation*} In particular, \eqref{30} and \eqref{40} imply \begin{equation*} \frac{y_{k+1}}{y_k}\le\exp(-4^k)\quad\text{and}\quad \frac{y_k}{y_{k-1}}\ge\exp(4^{k-1}). \tag{60}\label{60} \end{equation*}

Now recall \eqref{20} and write \begin{equation*} S=S_1+S_2+S_3, \tag{70}\label{70} \end{equation*} where \begin{equation*} S_1:=\sum_{i,j\colon k\le i<j}4^jy_iy_j, \quad S_2:=\sum_{i,j\colon 1\le i<j\le k}4^jy_iy_j, \quad S_3:=\sum_{i,j\colon i<k<j}4^jy_iy_j. \end{equation*} In view of \eqref{40}, \eqref{50}, and \eqref{60}, \begin{equation*} S_1\le c_1y_ky_{k+1}\le c_1 e^{-4^k}y_k^2, \end{equation*} where $c_1:=1+\sum_{j\ge1}j4^j e^{-2^j}\in(0,\infty)$; \begin{equation*} S_2\le k^2 4^k y_{k-1}y_k\le k^2 4^k e^{-4^{k-1}}y_k^2\le e^{-4^{k-2}}y_k^2, \end{equation*} since $k$ is large; \begin{equation*} S_3\le c_2 k y_{k-1}y_k\le e^{-4^{k-2}}y_k^2, \end{equation*} where $c_2:=\sum_{j\ge1}4^j e^{-2^j}\in(0,\infty)$.

So, by \eqref{20} and \eqref{70}, \begin{equation*} I\le e^{-4^{k-3}}y_k^2. \end{equation*} On the other hand, clearly, $J\ge y_k^2$. Thus, recalling \eqref{35}, \begin{equation*} V(h)=V(h_k)\le e^{-4^{k-3}}<e^{-c'2^{k+1}}=e^{-c'h_k} \end{equation*} for any real $c'>0$ and all large enough $k$. $\quad\Box$