Union Bound of two events?

Question

Inequality 1

\begin{align} \mathbb{P}\left(\frac{1}{n} \sum_{i=1}^{n}\left(f\left(x_{i}\right)-\mathbb{E}[f]\right) z_{i} \geq \frac{\epsilon}{8}\right) \leq 2 \exp \left(-\frac{\epsilon^{2} n d}{9^{4} c L^{2}}\right) \end{align}

Since we assumed that the range of the functions is in $[-1,1]$ we have $\mathbb{E}[f] \in[-1,1]$ and hence:

Inequality 2

$$ \mathbb{P}\left(\exists f \in \mathcal{F}: \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[f] z_{i} \geq \frac{\epsilon}{8}\right) \leq \mathbb{P}\left(\left|\frac{1}{n} \sum_{i=1}^{n} z_{i}\right| \geq \frac{\epsilon}{8}\right) $$

By Hoeffding's inequality, the above quantity is smaller than $2 \exp \left(-n \epsilon^{2} / 8^{3}\right)$ (recall that $\left.\left|z_{i}\right| \leq 2\right)$.

Thus we obtain with an union bound: $$ \begin{aligned} \mathbb{P}\left(\exists f \in \mathcal{F}: \frac{1}{n} \sum_{i=1}^{n} f\left(x_{i}\right) z_{i} \geq \frac{\epsilon}{4}\right) & \leq|\mathcal{F}| \cdot \mathbb{P}\left(\frac{1}{n} \sum_{i=1}^{n}\left(f\left(x_{i}\right)-\mathbb{E}[f]\right) z_{i} \geq \frac{\epsilon}{8}\right)+\mathbb{P}\left(\left|\frac{1}{n} \sum_{i=1}^{n} z_{i}\right| \geq \frac{\epsilon}{8}\right) \\ & \leq 2|\mathcal{F}| \cdot \exp \left(-\frac{\epsilon^{2} n d}{9^{4} c L^{2}}\right)+2 \exp \left(-\frac{n \epsilon^{2}}{8^{3}}\right) \end{aligned} $$

---

I am not getting how Union bound is getting happened using Ineq 1 and 2. Can anyone help me with that how they able to reach last inequality?

Answer 1

By the triangle inequality, the event $$A=\left\{\exists f \in \mathcal{F}: \frac{1}{n} \sum_{i=1}^{n} f\left(x_{i}\right) z_{i} \geq \frac{\epsilon}{4}\right\}$$ is contained in the union of the two events

$$B=\left\{\exists f \in \mathcal{F}: \frac{1}{n} \sum_{i=1}^{n}\left(f\left(x_{i}\right)-\mathbb{E}[f]\right) z_{i} \geq \frac{\epsilon}{8}\right\}$$ and $$D=\left\{\exists f \in \mathcal{F}: \ \frac{1}{n} \sum_{i=1}^{n} \mathbb{E}[f] z_{i} \geq \frac{\epsilon}{8}\right\} \,. $$ Moreover, $B=\bigcup_{f \in \mathcal{F}}B_f$ where $$B_f=\left\{ \frac{1}{n} \sum_{i=1}^{n}\left(f\left(x_{i}\right)-\mathbb{E}[f]\right) z_{i} \geq \frac{\epsilon}{8}\right\} \,, $$ so $$A\subset \left(\bigcup_{f \in \mathcal{F}} B_f \right) \cup B \,$$ To this relation the union bound is applied, together with inequality 2 to bound $P(D)$.