Question

Given a covariance matrix, how can I construct a vector of expressions of randomly distributed variables whose covariance matrix is equal to the given one?

EDIT: All variables are normally distributed.

I have an algorithm that gets the covariances correct, but not the variances on the diagonal:

a = [0]*len(r)
for x, row in enumerate(cov_matrix(r)):
    for y, item in enumerate(row):
        if x > y: continue
        v = noise(math.sqrt(abs(item)))
        a[x] += v
        if item > 0:
            a[y] += v
        else:
            a[y] -= v

I feel like this should be simple ...

Answer 1

If $A$ is your target covariance matrix and $LL^T = A$, and $x = (x_1, \ldots, x_n)$ is a vector of independent random variables with mean zero and variance 1, then $y = Lx$ has the required covariance. Here $L$ is a matrix and $L^T$ is its transpose. $L$ can just be the Cholesky factor of $A$. ((Check: $\mathrm{cov}(y) = E[yy^T] = E[(Lx)(Lx)^T] = E[Lxx^TL^T] = LE[xx^T]L^T$ (by linearity of expectation) $= L\mathrm{cov}(x)L^T = LIL^T = LL^T = A$. $\mathrm{cov}(y) = E[yy^T]$ because $y$ has mean 0, and likewise for $\mathrm{cov}(x)$.)

That's not too far from a "complete" solution, actually. If you start with a vector $y$ of random variables with mean zero and covariance matrix $A$, then if $A = LL^T$ and $x = L^{-1}y$, then $\mathrm{cov}(x) = I$. That doesn't necessarily imply that the components of $x$ are independent; it means they are uncorrelated. So the most general construction is to begin with a vector $x$ of uncorrelated random variables with mean zero and variance 1 and let $y=Lx$. (I only mean that every example can theoretically be obtained that way, not that it's necessarily the best or most computationally efficient way to do it.)

Answer 2

This question is perhaps more suited to stats exhange. Darsh suggested using the Cholesky decomposition, but this only works if the distribution of the random variables you want to generate is Gaussian. Otherwise there are two techniques that I know of, the Iman-Conover method and the methods based on Copulas.