Given a covariance matrix, how can I construct a vector of expressions of randomly distributed variables whose covariance matrix is equal to the given one?

EDIT: All variables are normally distributed.

I have an algorithm that gets the covariances correct, but not the variances on the diagonal:

a = [0]*len(r)
for x, row in enumerate(cov_matrix(r)):
    for y, item in enumerate(row):
        if x > y: continue
        v = noise(math.sqrt(abs(item)))
        a[x] += v
        if item > 0:
            a[y] += v
            a[y] -= v

I feel like this should be simple ...

| cite | improve this question | | | | |
  • $\begingroup$ Meta-question: Why did you write that as a comment instead of an answer? $\endgroup$ – Forrest Sep 4 '10 at 14:59
  • $\begingroup$ Good point. I was originally planning to leave a short comment, but I ended up giving more detail than I had thought I would. I deleted the comment and reposted it as an answer (and added a paragraph generalizing the construction slightly). $\endgroup$ – Darsh Ranjan Sep 4 '10 at 23:41

If $A$ is your target covariance matrix and $LL^T = A$, and $x = (x_1, \ldots, x_n)$ is a vector of independent random variables with mean zero and variance 1, then $y = Lx$ has the required covariance. Here $L$ is a matrix and $L^T$ is its transpose. $L$ can just be the Cholesky factor of $A$. ((Check: $\mathrm{cov}(y) = E[yy^T] = E[(Lx)(Lx)^T] = E[Lxx^TL^T] = LE[xx^T]L^T$ (by linearity of expectation) $= L\mathrm{cov}(x)L^T = LIL^T = LL^T = A$. $\mathrm{cov}(y) = E[yy^T]$ because $y$ has mean 0, and likewise for $\mathrm{cov}(x)$.)

That's not too far from a "complete" solution, actually. If you start with a vector $y$ of random variables with mean zero and covariance matrix $A$, then if $A = LL^T$ and $x = L^{-1}y$, then $\mathrm{cov}(x) = I$. That doesn't necessarily imply that the components of $x$ are independent; it means they are uncorrelated. So the most general construction is to begin with a vector $x$ of uncorrelated random variables with mean zero and variance 1 and let $y=Lx$. (I only mean that every example can theoretically be obtained that way, not that it's necessarily the best or most computationally efficient way to do it.)

| cite | improve this answer | | | | |
  • $\begingroup$ It depends on what you are doing. Say you want uniformly distributed random variables with a particular correlation structure. You can't just generate uncorrelated uniform random variables and apply the Cholesky decomposition because a sum of uniform random variables is no longer uniform. It will have the correct correlation (as you show) but the marginal distributions will not be uniform. $\endgroup$ – Robby McKilliam Sep 4 '10 at 23:47
  • $\begingroup$ The way I understood the problem, the only goal is to produce a random vector with a prescribed covariance matrix. If more information is prescribed, like the marginal distributions of the components, then I don't know if my method can be extended. $\endgroup$ – Darsh Ranjan Sep 5 '10 at 1:32

This question is perhaps more suited to stats exhange. Darsh suggested using the Cholesky decomposition, but this only works if the distribution of the random variables you want to generate is Gaussian. Otherwise there are two techniques that I know of, the Iman-Conover method and the methods based on Copulas.

| cite | improve this answer | | | | |
  • $\begingroup$ "Using [...] Cholesky decomposition [...] only works if the distribution of the random variables you want to generate is Gaussian": No (re-read Darsh's comment). $\endgroup$ – Did Sep 4 '10 at 6:29
  • $\begingroup$ @Didier: See my comment on Darsh's answer. $\endgroup$ – Robby McKilliam Sep 5 '10 at 2:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.