MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The mutual information $I(\mathfrak A_1;\mathfrak A_2)$ of two complete $\sigma$-algebras $\mathfrak A_1$ and $\mathfrak A_2$ in a Lebesgue probability space $(X,m)$ is the integral of the logarithm of the Radon-Nikodym derivative $dP/d(P_1\otimes P_2)$, where $P_i$ are the quotient measures on the factor-spaces $X_i$ determined by the $\sigma$-algebras $\mathfrak A_i$, respectively, and $P$ is the joint distribution on $X_1\times X_2$. If $\mathfrak A_i$ correspond to countable measurable partitions $\alpha_i$ of the space $X$, then $I(\mathfrak A_1;\mathfrak A_2)$ are expressed in terms of the associated entropies as $H(\alpha_1)+H(\alpha_2)-H(\alpha_1\vee\alpha_2)$. I need an explicit reference for the following fact (I know how to prove it): if $\mathfrak B_n$ is a decreasing sequence of $\sigma$-algebras converging to a limit $\mathfrak B$, and $\mathfrak A$ is another $\sigma$-algebra such that $I(\mathfrak A;\mathfrak B_1)<\infty$, then $I(\mathfrak A;\mathfrak B_n)$ converges to $I(\mathfrak A;\mathfrak B)$.

share|cite|improve this question
up vote 5 down vote accepted

The last proof of this continuity property (and of its analogue for increasing sequences) is given in this paper by Harremöes and Holst. It also contains a pretty comprehensive list of references to earlier work. Apparently, first this property was established by Pinsker in his 1960 book "Information and Information Stability of Random Variables and Processes".

share|cite|improve this answer

Mutual information is weak-* lower semicontinuous, because it is Kullback-Leibler divergence. For this see Pinsker's book or Dupuis-Ellis. This gives you the desired liminf. For the other direction (which is usually easy because I haven't used your monotonicity condition yet), maybe you have some monotonicity or convexity argument.

share|cite|improve this answer
Thank you - I know how to prove it. My question was about a reference. – R W Nov 6 '10 at 22:38

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.