# Correlation between 3 variables

For correlation measurement betweeen 2 variables, I use Pearson formula.

What formula can use to find degree of correlation between 3 variables ? My variabes are not symmetric: The correlation in question is between 1st variable and pair of the other two. But I don't have a formula to combine 2nd and 3rd into one variable. Variables have values -1, 0, 1, if it matters.

-
you could use multi-information, see Bell's "co-information lattice" www.rni.org/bell/nara4.pdf –  Yaroslav Bulatov Mar 9 '11 at 21:39
So, does $0$ correlate better with $(0,1)$ or with $(-1,1)$? Does $1$ correlate better with $(-1,1)$ or with $(0,0)$? I think you have to decide those questions first, in order to have any hope of doing what you want to do. –  Gerry Myerson Mar 10 '11 at 11:06
Gerry: Correlation is when changes in one var can be predicted looking at changes in another var(s). Consider 3 vectors X,Y,Z where correlation between (X,Y) is low, correlation between (X,Z) is low, but correlation exists between X and some function f of (Y,Z). Which methods help me to discover function f() ? –  Andrei Mar 11 '11 at 16:57

Maybe you need the theory of cumulants also called semi-invariants. For two random variables $X,Y$ the correlation (or second cumulant) is $v(X,Y)=E(XY)-E(X)E(Y)$ where $E$ denotes the expectation. Pearson's formula makes a dimensionless quantity $$r=\frac{v(X,Y)}{\sqrt{v(X,X) v(Y,Y)}}\ ,$$ i.e., $X$ and $Y$ might have units like centimeters but $r$ is a pure number. The third cumulant generalizes $v(X,Y)$ and measures a correlation of three variables `altogether', i.e., not indirectly resulting from their pairwise correlations. It is $$c(X,Y,Z)=E(XYZ)-E(X)E(YZ)-E(Y)E(XZ)-E(Z)E(XY)$$ $$+2E(X)E(Y)E(Z).$$ However I don't know what the natural or standard dimensionless analog of $r$ would be. A possibility is $$\frac{c(X,Y,Z)}{\sqrt{v(X,X)v(Y,Y)v(Z,Z)}}.$$ All this is about random variables, say discrete given by a finite sample $(x_i,y_i,z_i)$, $1\le i\le N$. Now in statistical estimation you might have things like $1/N$ turning into $1/(N-1)$ in the correct formulas to use.

-
@Abdelmalek: I think you have a typo? $v(X,Y)$ is the covariance, not the correlation. And "second cumulant" is normally used to refer to the variance of a random variable, not to the covariance of two random variables. –  Ian Martin Jun 19 '11 at 19:09
@Ian: I think this is just a matter of terminology which may differ according to one's background. Mine is in statistical physics. I use the word cumulant not only for a single random variable but also for collections of RVs with a given joint probability distribution. –  Abdelmalek Abdesselam Jun 20 '11 at 15:40
@Abdelmalek: Good to know. Thanks. –  Ian Martin Jun 20 '11 at 22:25