7
$\begingroup$

For correlation measurement betweeen 2 variables, I use Pearson formula.

What formula can use to find degree of correlation between 3 variables ? My variabes are not symmetric: The correlation in question is between 1st variable and pair of the other two. But I don't have a formula to combine 2nd and 3rd into one variable. Variables have values -1, 0, 1, if it matters.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ you could use multi-information, see Bell's "co-information lattice" www.rni.org/bell/nara4.pdf $\endgroup$
    – Yaroslav Bulatov
    Commented Mar 9, 2011 at 21:39
  • $\begingroup$ So, does $0$ correlate better with $(0,1)$ or with $(-1,1)$? Does $1$ correlate better with $(-1,1)$ or with $(0,0)$? I think you have to decide those questions first, in order to have any hope of doing what you want to do. $\endgroup$
    – Gerry Myerson
    Commented Mar 10, 2011 at 11:06
  • $\begingroup$ Gerry: Correlation is when changes in one var can be predicted looking at changes in another var(s). Consider 3 vectors X,Y,Z where correlation between (X,Y) is low, correlation between (X,Z) is low, but correlation exists between X and some function f of (Y,Z). Which methods help me to discover function f() ? $\endgroup$
    – Andrei
    Commented Mar 11, 2011 at 16:57
  • $\begingroup$ This is an intriguing question. It is rather surprising that over a 100 years after correlation among two variables was proposed and worked on (Fisher and Pearson), we (humanity) have not been able to come up with a measure of correlation among three variables. The answer is not easy, perhaps. Consider three variables. We then have three zero-order correlations among each pair of variables. It is also known that the three correlations are not completely independent of each other, in the sense that if two correlations are known, then (continued) $\endgroup$
    – dingus
    Commented Nov 6, 2017 at 20:34
  • $\begingroup$ the limits of the third is a direct function of the other two. Further, there may be suppression type relationships, that complicate things even further. $\endgroup$
    – dingus
    Commented Nov 6, 2017 at 20:34

3 Answers 3

Reset to default
8
$\begingroup$

Maybe you need the theory of cumulants also called semi-invariants. For two random variables $X,Y$ the correlation (or second cumulant) is $v(X,Y)=E(XY)-E(X)E(Y)$ where $E$ denotes the expectation. Pearson's formula makes a dimensionless quantity $$r=\frac{v(X,Y)}{\sqrt{v(X,X) v(Y,Y)}}\ ,$$ i.e., $X$ and $Y$ might have units like centimeters but $r$ is a pure number. The third cumulant generalizes $v(X,Y)$ and measures a correlation of three variables `altogether', i.e., not indirectly resulting from their pairwise correlations. It is $$ c(X,Y,Z)=E(XYZ)-E(X)E(YZ)-E(Y)E(XZ)-E(Z)E(XY) $$ $$ +2E(X)E(Y)E(Z). $$ However I don't know what the natural or standard dimensionless analog of $r$ would be. A possibility is $$ \frac{c(X,Y,Z)}{\sqrt{v(X,X)v(Y,Y)v(Z,Z)}}. $$ All this is about random variables, say discrete given by a finite sample $(x_i,y_i,z_i)$, $1\le i\le N$. Now in statistical estimation you might have things like $1/N$ turning into $1/(N-1)$ in the correct formulas to use.

Improve this answer
$\endgroup$
3
  • $\begingroup$ @Abdelmalek: I think you have a typo? $v(X,Y)$ is the covariance, not the correlation. And "second cumulant" is normally used to refer to the variance of a random variable, not to the covariance of two random variables. $\endgroup$
    – Ian Martin
    Commented Jun 19, 2011 at 19:09
  • 1
    $\begingroup$ @Ian: I think this is just a matter of terminology which may differ according to one's background. Mine is in statistical physics. I use the word cumulant not only for a single random variable but also for collections of RVs with a given joint probability distribution. $\endgroup$
    – Abdelmalek Abdesselam
    Commented Jun 20, 2011 at 15:40
  • $\begingroup$ @Abdelmalek: Good to know. Thanks. $\endgroup$
    – Ian Martin
    Commented Jun 20, 2011 at 22:25
2
$\begingroup$

I understand the question like the following example. First we consider the correlation of two variables, say age and income of professionals, and expect, that higher age agrees with higher income. Surely we have cases, where this is inverted: older professionals with lower income and/or younger professional with higher income.

Then we look at a third variable for instance political/ethical acceptance for that professional by other people, and may assume, that high ethical acceptance is high if age/income agree and acceptance is low if age/income disagree.

If such a constellation is asked for, then I would go back to the data and not to the aggregate's parameters. After z-standardizing of income and age I would construct an income/age-agree index agi by multiplying agi = z(income) x z(age) on case level. Then agi has high positive values if either age and income are high positive or if they both are high negative. Then I would correlate z(agi) with z(acceptance).

Improve this answer
$\endgroup$
1
$\begingroup$

Looking at your comment

correlation exists between X and some function f of (Y,Z). Which methods help me to discover function f() ?

it appears that what you mainly need is a good predictor of $X$ based on $(Y,Z)$. In terms of the least squares, the best predictor of $X$ based on $(Y,Z)$ is $E(X|Y,Z)$, the conditional expectation of $X$ given $(Y,Z)$; see e.g. Section "Best prediction", pages 3--4.

So, the optimal prediction function $f_*$ is given by the formula $f_*(y,z)=E(X|Y=y,Z=z)$ for $y$ and $z$ in the set $\{-1,0,1\}$. Then, if so desired, you can consider the correlation between $X$ and $f_*(Y,Z)$.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .