2
$\begingroup$

Suppose we are given two random variables $X$ and $Y$ with fixed marginal and joint distribution. What is the maximum randomness that we can extract from $Y$ that is independent from $X$, that is, if $Z=f(Y)$ is a transformation of $Y$, what is $\sup_f H(Z)$ where supremum is taken over all functions $f$ for which $f(Y)$ and $X$ are independent.

A really loose bound for this is $H(Y|X)$. This is loose, because for example if $X$ and $Y$ are dependent binary random variables with some joint distribution, independence of $Z$ and $X$ yields independence of $Z$ and $Y$, however, $H(Y|X)$ is not necessarily not zero.

A natural extension of this, is to relax the condition of "$Z$ and $X$ being independent" to "$Z$ and $X$ being $\epsilon$-independent", which means, mutual information $I(Z;X)\leq \epsilon$ or $|P(Z, X)-P(Z)P(V)|\leq\epsilon$ or any other notion of $\epsilon$-independence that you can think of for making this problem easier..

I would be really thankful if you can give any idea on this,

$\endgroup$
3
  • 1
    $\begingroup$ This is an entire area of study (and I'm kind of surprised you're asking such a broad question because you seem to know most of the terminology already). It would be nice if someone with more expertise than I can summarize the state of the art for you. I can at least point to Vadhan's book on Pseudorandomness which has a chapter on extractors: people.seas.harvard.edu/~salil/pseudorandomness $\endgroup$
    – usul
    Commented Feb 7, 2014 at 6:27
  • $\begingroup$ @usul, thanks for the comment and nice reference you sent me, as far as I could get from this chapter, the output of each extractor should be close to uniform distribution and also all sources are assumed to be binary. However, in this problem, all sources lie in arbitrary spaces, and also output of $f$ are not necessarily uniform binary. $\endgroup$ Commented Feb 7, 2014 at 19:51
  • $\begingroup$ Good point. I wish I had the knowledge to give a good answer! (I guess there is some naive transformation that works "ok", if we think of Y and Z as being encoded by bitstrings and if we can sample from Z by running a function on uniformly random inputs...) $\endgroup$
    – usul
    Commented Feb 7, 2014 at 23:39

0

You must log in to answer this question.

Browse other questions tagged .