# Correlation between 3 variables

For correlation measurement betweeen 2 variables, I use Pearson formula.

What formula can use to find degree of correlation between 3 variables ? My variabes are not symmetric: The correlation in question is between 1st variable and pair of the other two. But I don't have a formula to combine 2nd and 3rd into one variable. Variables have values -1, 0, 1, if it matters.

-
you could use multi-information, see Bell's "co-information lattice" www.rni.org/bell/nara4.pdf –  Yaroslav Bulatov Mar 9 '11 at 21:39
So, does $0$ correlate better with $(0,1)$ or with $(-1,1)$? Does $1$ correlate better with $(-1,1)$ or with $(0,0)$? I think you have to decide those questions first, in order to have any hope of doing what you want to do. –  Gerry Myerson Mar 10 '11 at 11:06
Gerry: Correlation is when changes in one var can be predicted looking at changes in another var(s). Consider 3 vectors X,Y,Z where correlation between (X,Y) is low, correlation between (X,Z) is low, but correlation exists between X and some function f of (Y,Z). Which methods help me to discover function f() ? –  Andrei Mar 11 '11 at 16:57

Maybe you need the theory of cumulants also called semi-invariants. For two random variables $X,Y$ the correlation (or second cumulant) is $v(X,Y)=E(XY)-E(X)E(Y)$ where $E$ denotes the expectation. Pearson's formula makes a dimensionless quantity $$r=\frac{v(X,Y)}{\sqrt{v(X,X) v(Y,Y)}}\ ,$$ i.e., $X$ and $Y$ might have units like centimeters but $r$ is a pure number. The third cumulant generalizes $v(X,Y)$ and measures a correlation of three variables `altogether', i.e., not indirectly resulting from their pairwise correlations. It is $$c(X,Y,Z)=E(XYZ)-E(X)E(YZ)-E(Y)E(XZ)-E(Z)E(XY)$$ $$+2E(X)E(Y)E(Z).$$ However I don't know what the natural or standard dimensionless analog of $r$ would be. A possibility is $$\frac{c(X,Y,Z)}{\sqrt{v(X,X)v(Y,Y)v(Z,Z)}}.$$ All this is about random variables, say discrete given by a finite sample $(x_i,y_i,z_i)$, $1\le i\le N$. Now in statistical estimation you might have things like $1/N$ turning into $1/(N-1)$ in the correct formulas to use.

-
@Abdelmalek: I think you have a typo? $v(X,Y)$ is the covariance, not the correlation. And "second cumulant" is normally used to refer to the variance of a random variable, not to the covariance of two random variables. –  Ian Martin Jun 19 '11 at 19:09
@Ian: I think this is just a matter of terminology which may differ according to one's background. Mine is in statistical physics. I use the word cumulant not only for a single random variable but also for collections of RVs with a given joint probability distribution. –  Abdelmalek Abdesselam Jun 20 '11 at 15:40
@Abdelmalek: Good to know. Thanks. –  Ian Martin Jun 20 '11 at 22:25

I understand the question like the following example. First we consider the correlation of two variables, say age and income of professionals, and expect, that higher age agrees with higher income. Surely we have cases, where this is inverted: older professionals with lower income and/or younger professional with higher income.

Then we look at a third variable for instance political/ethical acceptance for that professional by other people, and may assume, that high ethical acceptance is high if age/income agree and acceptance is low if age/income disagree.

If such a constellation is asked for, then I would go back to the data and not to the aggregate's parameters. After z-standardizing of income and age I would construct an income/age-agree index agi by multiplying agi = z(income) x z(age) on case level. Then agi has high positive values if either age and income are high positive or if they both are high negative. Then I would correlate z(agi) with z(acceptance).

-