Let $\psi_\alpha(x) := \exp(x^\alpha)-1$. The Sub-Gaussian Norm $\lVert X \rVert_{\psi_2}$ of a random variable $X$ is defined as
$$ \lVert X\rVert_{\psi_2} = \inf\{c>0\mid \mathbb{E}[\varphi_2(|X|/c)] \leq 1\}, $$ where one takes the convention that $\lVert X\rVert_{\psi_2} = \infty$ if this set is empty. One may analogously define the sub-Exponential norm $\lVert X\rVert_{\psi_1}$.
These are norms on the space of random variables, so they satisfy all of the identities that norms do. They satisfy the additional identities
- $\lVert X^2\rVert_{\psi_1} = \lVert X\rVert_{\psi_2}^2$
- $\lVert XY\rVert_{\psi_1} \leq\lVert X\rVert_{\psi_2}\lVert Y\rVert_{\psi_2}$,
- $\lVert XY\rVert_{\psi_2} \leq \lVert X\rVert_{\psi_2}\lVert Y\rVert_{\infty}$, and
- $\lVert \sum_i X_i\rVert_{\psi_2}^2 \leq \sum_i \lVert X_i\rVert_{\psi_2}^2$, provided the $X_i$ are independent.
I particularly like inequality 3, as it allows one to start with a $\psi_2$ bound on $X$, and yields a $\psi_2$ bound on $XY$, under the (restrictive) assumption that $Y$ is bounded.
I'm curious if it can be extended to unbounded $Y$. As an example, if $\lVert Y\rVert_{\psi_2} < \infty$, standard computations show that
$$ \lVert Y\rVert_\infty \leq \sqrt{2\ln(1/\delta)}\lVert Y\rVert_{\psi_2}\text{ except with probability }\leq \delta. $$
So $Y$ is "bounded with high probability". One could then apply Eq. (3), to get that
$$ \lVert XY\rVert_{\psi_2} \leq \sqrt{2\ln(1/\delta)}\lVert X\rVert_{\psi_2}\lVert Y\rVert_{\psi_2},$$ except with probability $\delta$. By this, I mean there exists some set $A$ such that for $p_Y(x)$ the PDF of $Y$, we have that $p_{Y_A}(x) := \frac{p_Y(x)\mathbb{I}_A(x)}{\Pr[Y\in A]}$, where $\mathbb{I}_A(x)$ is the indicator function of $x\in A$, satisfies $$ \lVert XY_A\rVert_{\psi_2} \leq \sqrt{2\ln(1/\delta)}\lVert X\rVert_{\Psi_2}\lVert Y\rVert_{\psi_2}, $$ and $\Pr[A] \leq \delta$. This yields a weakened sub-Gaussian tail-bound
$$\tag{1} \Pr[|XY| \geq \sqrt{2\ln(1/\delta)}\lVert X\rVert_{\psi_2}\lVert Y\rVert_{\psi_2}t] \leq \exp(-t^2) + \delta, $$ which would still be useful to me.
This is all fine so far (and easily provable). What I'm curious is if this type of reasoning composes. In particular, if one can obtain some bound of the form of Eq.(1) for 3 multiplications (for example) by iterated conditioning $(XY_A)_BZ$. By choosing $B$ appropriately (to be the event that $XY_A$ is bounded by $\sqrt{2\ln(1/\delta)}\lVert X\rVert_{\psi_2}\lVert Y_A\rVert_{\infty} \leq 2\ln(1/\delta)\lVert X\rVert_{\psi_2}\lVert Y\rVert_{\psi_2}$), one hopes to get the bound $$ \Pr[|XYZ| \geq \sqrt{2\ln(1/\delta)}^2\lVert X\rVert_{\psi_2}\lVert Y\rVert_{\psi_2}\lVert Z\rVert_{\psi_2}] \leq \exp(-t^2)+2\delta. $$
This seems reasonable to me, but I'm both not a probabilist, and know that it is quite easy to incorrectly reason about conditional probability. This leads to the questions
- Can one extend sub-gaussian analysis to heavier-tailed random variables by appropriately conditioning on the events that these random variables are bounded? I'd be especially interested to see references to prior work where this type of argument occurs.
- If so, is the iterated conditioning argument referenced to above as straightforward as I am hoping?
The answer I'm hoping for is "this is a standard argument, see [some citation] for how to properly do it". I haven't been able to find such a citation. I've seen that such truncation arguments are common, but not how they interact with standard sub-Gaussian analysis (if they do at all). I don't know if that is because this is a specialized question, or if it is known to not work out well.