# Axioms for mutual information

I am interesting in axiomatic justifications for concepts in information theory. I have found many axiomatizations for Shannon's entropy and for the Kullback-Leibler divergence, as well as their variations (for some examples, see this survey or these books: 1, 2). However, I have not found any results directly for the mutual information.

Are there any results that show that some intuitive properties uniquely characterize the mutual information of a joint distribution of two random variables?

• Can I rephrase this as : Given a random sequencegenerated by a random variable X and another by a random variable Y, is their an algorithm to find the mutual information of X and Y. You want an axiomatic justification of info theory argument?
– ARi
Jun 19, 2013 at 5:44
• I think I have in mind something different. All I'm saying is: look at the mutual information as a function from pairs of random variables to real numbers. I'm looking for a theorem of the form: a function (with this domain and codomain) satisfies properties 1 to n if and only if it's the mutual information of the two random variables. Jun 19, 2013 at 19:02
• I'm not sure if I've seen such a characterization before, but since mutual information is a KL divergence, it might be possible to add axioms to those for the KL divergence... Aug 23, 2013 at 20:38

Our main result is Theorem~6.6, where we prove that the mutual information $$\mathbb{I}(\mathcal{X},\mathcal{Y})$$ of an ordered pair of random variables is the unique function (up to an arbitrary multiplicative factor) on pairs of random variables satisfying the following axioms: