1
$\begingroup$

In my studies of applied probability I have recently met the following problem on which I need help:

We consider two binary random (column) vectors $ X,Y \in \{0,1\}^d $ where the mutual distribution satisfies $ P(X,Y) = \frac{e^{-X^TWY-b^TX-c^TY}}{Z(W,b,c)} $ where W is a symmetric positive semidefinite matrix, b and c are constant column vectors of length $ d $ and $ Z $ is the partition function to normalize the probability distribution, this setting might be familiar to some from machine learning and neural networks as a "Restricted Boltzmann Machine" (RBM), I was looking at the mutual information $ I(X;Y) = \sum_{X} \sum_{Y} P(X,Y) \ln{(\frac{P(X,Y)}{P(X)P(Y)})}$

I was wondering if we can approximate this quantity (the mutual information) or even calculate it analytically? We may assume the vectors are over {-1,1} if it makes things easier. Maybe an asymptotic approximation would help? I was thinking maybe a probabilistic approximation, maybe approximating the discrete distributions as Gaussian distributions or other distributions easier to deal with? I thank all helpers on this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.