8
$\begingroup$

I am interesting in axiomatic justifications for concepts in information theory. I have found many axiomatizations for Shannon's entropy and for the Kullback-Leibler divergence, as well as their variations (for some examples, see this survey or these books: 1, 2). However, I have not found any results directly for the mutual information.

Are there any results that show that some intuitive properties uniquely characterize the mutual information of a joint distribution of two random variables?

Improve this question
$\endgroup$
3
  • $\begingroup$ Can I rephrase this as : Given a random sequencegenerated by a random variable X and another by a random variable Y, is their an algorithm to find the mutual information of X and Y. You want an axiomatic justification of info theory argument? $\endgroup$
    – ARi
    Commented Jun 19, 2013 at 5:44
  • $\begingroup$ I think I have in mind something different. All I'm saying is: look at the mutual information as a function from pairs of random variables to real numbers. I'm looking for a theorem of the form: a function (with this domain and codomain) satisfies properties 1 to n if and only if it's the mutual information of the two random variables. $\endgroup$
    – Henrique de Oliveira
    Commented Jun 19, 2013 at 19:02
  • 1
    $\begingroup$ I'm not sure if I've seen such a characterization before, but since mutual information is a KL divergence, it might be possible to add axioms to those for the KL divergence... $\endgroup$
    – Anand Sarwate
    Commented Aug 23, 2013 at 20:38

1 Answer 1

Reset to default
3
$\begingroup$

My paper https://arxiv.org/pdf/2108.12647.pdf provides an answer to this question. In particular, here is a quote from the introduction:

Our main result is Theorem~6.6, where we prove that the mutual information $\mathbb{I}(\mathcal{X},\mathcal{Y})$ of an ordered pair of random variables is the unique function (up to an arbitrary multiplicative factor) on pairs of random variables satisfying the following axioms: enter image description here

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .