Suppose $\mathcal{A},\mathcal{B}$ are finite sets and $\mu_{A,B}(a,b)$ is a probability measure on the product set $\mathcal{A}\times \mathcal{B}$ so that $\mu_{A,B}(a,b)>0$ for each $a\in \mathcal{A}, b\in \mathcal{B}.$ We may tilt the measure $\mu$ by choosing non-negative (not everywhere zero) functions $\phi(a), \psi(b)$ defining a new measure:


We restrict ourselves to such tilted measures and ask the questions:

a) Given marginal probability measures $\nu_A(a), \nu_B(b),$ is there always a tilted measure $\nu_{A,B}(a,b)$ that has these marginals?

b) If the answer to a) is yes, is there a unique tilted measure that has these marginals?

The answer to (a) is indeed YES, since we can consider the relative entropy minimization problem:

$$\arg\min_{\tilde{\nu}_A = \nu_A, \tilde{\nu}_B = \nu_B} \sum_{a,b} \tilde{\nu}_{A,B}(a,b)\log\frac{\tilde{\nu}_{A,B}(a,b)}{\mu_{A,B}(a,b)}.$$

This is a convex programming problem with a unique minimizer that can be shown using Lagrange multipliers to be of the tilted measure form.

My questions then are:

(b): Given any pair of marginals, is there a unique tilted measure with these marginals?

(c): Is there a name for tilted measures of the product form as I have considered here?

I suspect the answer to (b) is YES and this suspicion comes from the fact that the space of pairs of marginals has dimension $(|\mathcal{A}|+|\mathcal{B}|-2)$ which is also the dimension of the space of tilted measures.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.