If one is given a joint probability distribution over a finite set of discrete random variables then I guess there a notion of $\alpha-$Renyi entropy defined for it as $S_\alpha (X_1,..,X_n) = \frac{1}{1-\alpha} \log ( \sum_{a_1,..,a_n} p_{X_1,..,X_n}^\alpha (X_1 = a_1,..,X_n = a_n ) )$

  • Can the sum in the above is read as taking the trace of the $\alpha$ power of some density matrix?

  • Lets say the joint probability distribution is given as a probabilistic graphical model. Now given such a graph one can split it into sub-graphs and look at the joint distributions of each of these subgraphs. So one presumably has the $S_\alpha$ defined for each of these subgraphs. Do we have any relation between $S_\alpha (graph)$ and $\sum_{subgraphs} S_\alpha (subgraphs)$? (consider that the decomposition could be into disjoint parts or not)


The answer to your first question is "yes, but such a matrix may not necessarily have an operational significance, until guessing strategies come in." A good starting point may be the paper Revisiting Conditional Renyi Entropies ... which seems to have a good discussion of the non uniqueness of conditional Renyi entropies, and is publicly accessible.

If you are in addition imposing a graphical probability model, you will need to develop this kind of theory further, in the light of your goals.

| cite | improve this answer | |
  • $\begingroup$ Why do you say these are "conditional Renyi entropies"? Also could you describe what is the density matrix you have in mind and what are these guessing strategies? $\endgroup$ – Anirbit Mar 31 '16 at 3:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.