Question

We always hear, when reading on correlation, that "correlation does not imply causation."

Still, I have never seen any source that tries to answer the question of when can we reasonably conclude a causal relation between variables X, Y from a correlation.

After talking about it with some friends, we conclude these factors are at least necessary, if not sufficient to conclude causation:

1. A high value of r², with r the correlation coefficient. Therefore, the value of r itself is also close to ±1.

2. Data on X, Y was obtained under controlled/experimental conditions, and data for X, Y, taken under similar experimental conditions produces similar values for r².

3. The time interval t between the independent X and the independent Y in many trials is short (if not immediate), and the values of t from different trials fall in a narrow interval.

Anyone have comments?

Answer 1

Many scientific disciplines are such that experiments are impossible. The effects of childhood abuse, for example. Scientists who study such matters are generally very sensitive to the fact that the empirical work they do cannot prove causation in the way that controlled studies can.

What a well-designed correlative study can prove or disprove is that of a long list of proposed explanations for some effect, one is the most likely. For example, you can make a study that determines which of the claims "A causes C" or "B causes C" is more likely. If you do enough of these, against every conceivable other thing that might effect C, then you can reasonably assert that you have evidence that A causes C, but you have to remember that it is a different kind of evidence from in-lab controlled experiments. (Sometimes it is better evidence: lab environments can be poor approximations of the actual world.) ((Yet another kind of evidence would be a proposed mechanism. If you can tell a convincing yarn about why A causes C, which builds on a series of well-established causal relationships, then you may claim you have evidence for your causal assertion. I think that this kind of evidence is generally the weakest, but it is often the one that people like the best, since most people understand the world through stories.))

But it is important to remember that just the existence of a correlation need not have much to do with a causal relationship. Here's one that comes to mind. It is a fact that a history of childhood victimization predicts for (is correlated with) lower adulthood weight (the effect is weak, but statistically significant). However, the best guess for the relationship between childhood victimization and weight is that victimization causes the victim to be overweight in adulthood. This result shows up in the empirical data after you "hold other variables constant". In particular, childhood victimization strongly correlates with tobacco use, and tobacco is known to make people thinner. But if you child-abuse-victimization with weight within either the tobacco-using population or the tobacco-non-using population, you do see a correlation between abuse and obesity. Again, it is always possible that there are other better explanations for this effect, and to test them you go out in the field and measure all your proposed variables. My understanding of the result "child abuse causes obesity" is that it is more likely than any other explanation that people have thought of, in the sense that every variable scientists have thought of to hold constant or covary or all the other things they can do in the statistical models seems to lead to the asserted conclusion.

Answer 2

Learning causal relationships (i.e. a directed acyclic graph where "$A \to B$ means $A$ causes $B$") from observational data is a kind of causal inference. In general, it is not possible. However, the following two conditions, phrased in the terminology of graphical models, are sufficient for models without confounding variables.

• Causal Markov assumption: each node is conditionally independent of its non-effects, direct or otherwise, given its direct causes (i.e. parents).
• Faithfulness: the only conditional independencies in the true distribution arise from d-separation in the true causal DAG.

Judea Pearl, Clark Glymour, and Peter Spirtes have done excellent work in this area. Since you are asking your question on MO, you are presumably interested in mathematically-oriented discussion, such as the one in Koller and Friedman's Probabilistic Graphical Models.

Answer 3

First, 1) is certainly not necessary, since correlation can be 0 while Y is a function of X. Indeed, correlation detects only affine relations between X and Y; see wikipedia http://en.wikipedia.org/wiki/Correlation_and_dependence for examples and a discussion on the relation between correlation and dependence.

Second, the classical example to have in mind when adressing this question is the case when there is a hidden variable Z, whose value imply (in the sense of causality) that of X and Y. Then, it can happen that X and Y are entirely dependent (although they could very well be independent) but neither X imply Y nor Y imply X.

Also, note that logical implication need not follow from causality (if you have a wet umbrella in your hand, I can deduce that it rained, however it is not your umbrella that caused the rain).

Last, I would say that causality seems not to be a mathematical matter, not at least in the domain of probability and statistics. To establish causality, you usually need some knowledge of how the world works.

Answer 4

You might want to look at tests for Granger causality, which formalise the idea that "X causes Y if X occurs before and is correlated with Y". The tests suffer from the same problem mentioned elsewhere: they can't rule out a third event or process Z which causes both X and Y.

Answer 5

Item 2 in your list is the big one.

There is a very extensive literature on this, and I'm not really familiar with it, but with Google scholar and Google books you should find plenty.

A large r^2 is absolutely unnecessary. An r^2 value of 0.00001 can be overwhelmingly statistically significant if the sample size is very large.

Answer 6

Correlation (assuming its existence is correctly detected) does imply shared dependency on one or more other variables. From enough data you can also detect whether $Y$ is (close enough to being) a function of $X$ alone, and absence of causation, where some sets of variables are independent of others. Beyond that you need to consider specific mechanical models of what-causes-what and the data available may or may not be enough to answer that. As others mentioned there are formal analyses of this problem by people doing statistics and machine learning, e.g., in Pearl's book.

Answer 7

One approach is to say that X causes Y if the shortest description (Kolmogorov complexity) of P(X,Y) consists of separate description of P(X) and a deterministic function f:X->Y. Since Kolmogorov complexity is uncomputable, it's replaced by more practical notions of description length. See Inferring deterministic causal relations from UAI'10 for an example

Answer 8

I apologize if this is not the format for replying to comments made on my post; the "comments" option was disabled.

Dorais' comments seem condescending and unfair. If you are "too big" for my "little question",dorais, and you do not, out of basic fairness, want to make
an effort to assume my question makes sense before giving a snide reply, then spare me your answer-- however many experiments you have done,francois.
I am interested in a fair debate, not in your cheap shots.

Herb