# Continuity of the mutual information

The mutual information $I(\mathfrak A_1;\mathfrak A_2)$ of two complete $\sigma$-algebras $\mathfrak A_1$ and $\mathfrak A_2$ in a Lebesgue probability space $(X,m)$ is the integral of the logarithm of the Radon-Nikodym derivative $dP/d(P_1\otimes P_2)$, where $P_i$ are the quotient measures on the factor-spaces $X_i$ determined by the $\sigma$-algebras $\mathfrak A_i$, respectively, and $P$ is the joint distribution on $X_1\times X_2$. If $\mathfrak A_i$ correspond to countable measurable partitions $\alpha_i$ of the space $X$, then $I(\mathfrak A_1;\mathfrak A_2)$ are expressed in terms of the associated entropies as $H(\alpha_1)+H(\alpha_2)-H(\alpha_1\vee\alpha_2)$. I need an explicit reference for the following fact (I know how to prove it): if $\mathfrak B_n$ is a decreasing sequence of $\sigma$-algebras converging to a limit $\mathfrak B$, and $\mathfrak A$ is another $\sigma$-algebra such that $I(\mathfrak A;\mathfrak B_1)<\infty$, then $I(\mathfrak A;\mathfrak B_n)$ converges to $I(\mathfrak A;\mathfrak B)$.

## 2 Answers

The last proof of this continuity property (and of its analogue for increasing sequences) is given in this paper by Harremöes and Holst. It also contains a pretty comprehensive list of references to earlier work. Apparently, first this property was established by Pinsker in his 1960 book "Information and Information Stability of Random Variables and Processes".

Mutual information is weak-* lower semicontinuous, because it is Kullback-Leibler divergence. For this see Pinsker's book or Dupuis-Ellis. This gives you the desired liminf. For the other direction (which is usually easy because I haven't used your monotonicity condition yet), maybe you have some monotonicity or convexity argument.

• Thank you - I know how to prove it. My question was about a reference. – R W Nov 6 '10 at 22:38