Let $(\Omega,\mathcal{F},P)$ be a probability space, $\{\mathcal{F}_n \subseteq \mathcal{F}\}_{n \in \mathbb{N}}$ a filtration and $X: \Omega \rightarrow \mathbb{N}$ a random variable. Denote $\mathcal{P}(\Omega)$ the space of probability measures on $(\Omega, \mathcal{F})$ and assume that $\{Q_n: \Omega \rightarrow \mathcal{P}(\Omega)\}_{n \in \mathbb{N}}$ are regular conditional probabilities corresponding to $\mathcal{F}_n$. For each $n \in \mathbb{N}$, define the *information gain* $I_n: \Omega \rightarrow \mathbb{R} \sqcup \{+\infty\}$ by

$$I_n(\omega):= D_{KL}(X_* Q_{n+1}(x) \mid\mid X_* Q_{n}(x))$$

Here, $X_*$ denotes push-forward by $X$ (so that $X_* Q_{n}(x)$ is a probability measure on $\mathbb{N}$).

> Is it true that $\sum_{n=0}^\infty I_n < \infty$ almost surely?

If it helps, consider any or all of the following simplifying assumptions:

* $\Omega$ is standard.
* $\mathcal{F}_n$ corresponds to a finite partition of $\Omega$.
* $X$ has finite range.