Constructing Bernoulli random variables with prescribed correlation - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-19T06:37:53Z http://mathoverflow.net/feeds/question/19406 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/19406/constructing-bernoulli-random-variables-with-prescribed-correlation Constructing Bernoulli random variables with prescribed correlation Alekk 2010-03-26T11:37:04Z 2011-11-08T23:49:40Z <p>For which $n \times n$ correlation matrix $C$ can one construct Bernoulli random variables $(B_1, \ldots, B_n)$ with correlation $C$ ?</p> <p>Following the approach described in this MO <a href="http://mathoverflow.net/questions/18268/discrete-stochastic-process-exponentially-correlated-bernoulli" rel="nofollow">thread</a>, one can think of the following construction. Define independent Bernoulli random variables $B_{k_1, \ldots, k_n}$ for $(k_1, \ldots, k_n) \in \mathbb{Z}^n$ and another independent $\mathbb{Z}^k$-valued random variable $I=(I_1, \ldots, I_n)$. Then $(B_{I_1}, \ldots, B_{I_n})$ is a correlated Bernoulli vector.</p> <p>1: Is there any example of correlation structure that cannot be obtained this way ?</p> <p>2: Any easy example of correlation matrix $C$ that cannot be the correlation matrix of a $\{0,1\}^n$ valued random vector ?</p> http://mathoverflow.net/questions/19406/constructing-bernoulli-random-variables-with-prescribed-correlation/19436#19436 Answer by gowers for Constructing Bernoulli random variables with prescribed correlation gowers 2010-03-26T15:36:47Z 2010-03-26T21:56:03Z <p>Here's a pretty general construction. Take unit vectors $v_1,\dots,v_n$ in $\mathbb{R}^n$ and let $u$ be a random unit vector (chosen with the uniform probability measure on the unit sphere). Define $B_i$ to be 1 if the inner product of $u$ and $v_i$ is positive and -1 otherwise. Then the correlation between $B_i$ and $B_j$ is the inner product of $v_i$ and $v_j$. (I haven't checked that carefully but I think it's true.)</p> <p>Added in the light of the comment below: OK, I should have checked. It's actually not the inner product but &pi; minus twice the angle between the two vectors all over &pi;. I.e., it depends linearly on the angle between the two vectors, is 1 when that angle is zero and -1 when it is &pi;. The angle is the inverse cos of the inner product, which gives us a formula. </p> <p>So it gives us a fairly big supply of matrices -- I can't quite decide whether it gives us all (or rather all for which the variables take two values, each with probability 1/2).</p> <p>A more general-looking construction is this. Take any probability space and let $A_1,...,A_n$ be sets of measure 1/2. Pick a random point x and let $B_i$ be 1 if x is in $A_i$ and -1 otherwise. But that becomes trivial, because if you have any set of Bernoulli variables taking the values $\pm 1$ with probability 1/2, then you can set $A_i$ to be the set where $B_i=1$.</p> http://mathoverflow.net/questions/19406/constructing-bernoulli-random-variables-with-prescribed-correlation/20550#20550 Answer by Didier Piau for Constructing Bernoulli random variables with prescribed correlation Didier Piau 2010-04-06T21:34:36Z 2010-04-07T05:57:27Z <p>Re 1: Not quite sure that I understand your construction but apparently it can never yield negative correlations?</p> <p>Re 2: Consider the matrix $C_3=\pmatrix{1&amp;-1/2&amp;-1/2\cr -1/2&amp;1&amp;-1/2\cr -1/2&amp;-1/2&amp;1}$. </p> <p>This is a legitimate correlation matrix (for instance of the Gaussian vector $(N,-N/2+N'\sqrt3/2,-N/2-N'\sqrt3/2)$ with $(N,N')$ i.i.d. and Gaussian) but neither $C_3$ nor any matrix whose $C_3$ is a submatrix are correlation matrices of Bernoulli vectors.</p> <p>To see this, assume that $C_3$ is the correlation matrix of a random vector $X=(X_1,X_2,X_3)$ with values in <code>$\{0,1\}^3$</code> and for every $i$ let $x_i^2=E(X_i)=E(X_i^2)$. Then $X_1/x_1+X_2/x_2+X_3/x_3$ is almost surely constant because the sum of the coefficients of $C_3$ is $0$. Hence $X_1/x_1+X_2/x_2$ takes exactly two values. For non degenerate <code>$\{0,1\}$</code> valued random variables $X_1$ and $X_2$, this means that $x_1=x_2$. Likewise, $x_1=x_3$, hence $S=X_1+X_2+X_3$ is almost surely constant. Now $S=0$ or $S=3$ means that $X=(0,0,0)$ or that $X=(1,1,1)$, respectively, hence these cases are excluded. By the symmetry $X_i\to1-X_i$, one can assume that $S=1$ almost surely. This means that $X$ is concentrated on the three points $(1,0,0)$, $(0,1,0)$ and $(0,0,1)$ and, furthermore (recalling the relations $x_1=x_2=x_3$), that $X$ is uniformly distributed on these three points. Thus the correlation of $X_1$ and $X_2$ is $-1/3$ and not $-1/2$ as it should be.</p> <p><b>Edit:</b></p> <p>I guess the same reasoning excludes every $n\times n$ matrix $C_n$ with diagonal entries $1$ and off-diagonal entries $-1/(n-1)$, for $n\ge3$.</p> <p>By the way, one sees that $C_3$ cannot be obtained through Gaussian random variables and hyperplanes as in Gowers' answer because, if it was, it would be produced by the $3\times3$ matrix with diagonal entries $\sin(1\cdot\pi/2)=1$ and off-diagonal entries $\sin((-1/2)\cdot\pi/2)=-1/\sqrt2$, which is not definite positive. (The same applies to $C_n$ for every $n\ge3$.)</p> <p>Correlation matrices $C=(C_{i,j})$ of Bernoulli random vectors might be exactly those such that the matrix $(\sin(C_{i,j}\pi/2))$ is definite positive, in which case the Gaussian-cut-by-hyperplanes construction would yield them all.</p> http://mathoverflow.net/questions/19406/constructing-bernoulli-random-variables-with-prescribed-correlation/80437#80437 Answer by yori for Constructing Bernoulli random variables with prescribed correlation yori 2011-11-08T23:49:40Z 2011-11-08T23:49:40Z <p>I have a question about Didier Piau's answer: Why does it follow that "the correlation of $X_1$ and $X_2$ is $−1/3$ and not $−1/2$ as it should be"? </p> <p>Perhaps I misunderstood the setting, but if $X$ is uniformly distributed over the points $(1,0,0)$, $(0, 1, 0)$ and $(0,0,1)$, then $E(X_i) = 1/3$, $E(X_iX_j)=0$, and hence $Var(X_i) = 2/9$ and $Cov(X_iX_j) = -1/9$. Therefore, the correlation between $X_i$ and $X_j$ is $(-1/9)/(2/9) = -1/2$, as it should be. In fact I think that, in this setting, the correlation matrix of $X_1, X_2, X_3$ is exactly the matrix $C_3$.</p>