4
$\begingroup$

Consider 2 independent random variables $X$ and $Y$ with values in $A=\{0, 1, \ldots, q-1\}$. Suppose that $P(X=Y)$ is equal to some constant $\varepsilon$.

  1. What is the maximal entropy $H(X, Y)$?

  2. At which distribution this maximum is attained?

My hypothesis is that the distribution $P(X=0)=P(Y=0)=\alpha$, $P(X=k)=P(Y=k)=(1-\alpha) / (q - 1)$, $k=1,\ldots,q-1$ gives the optimal answer, where $\alpha$ is calculated from equation $\alpha^2+(1-\alpha)^2/(q-1)=\varepsilon$. Using the standard method of Lagrange multipliers and some hacks I've managed to prove that for $\varepsilon>1/q$ the optimal distribution have the following form $P(X=i)=P(Y=i)=p_1$ for $i=0, \ldots, l-1$ and $P(X=i)=P(Y=i)=p_2$ for $k=l,\ldots,q-1$.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Your hypothesis sounds reasonable for $\varepsilon > 1/q$, but since you construct $X$ and $Y$ to be identically distributed, won't this always fail if $\varepsilon < 1/q$ since then (for iid random variables), one has $P(X=Y) \geq 1/q$? In other words, the equation for alpha has no real solution in this case. (Or is your hypothesis only for the case $\varepsilon > 1/q$?) $\endgroup$
    – Steve
    Commented Mar 2, 2018 at 20:29

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .