2
$\begingroup$

So for me the definition the independence of two random variables $X,Y$ is intuitivly very clear.

But what I have never seen motivated is why the heck one would be interested in the covariance $$\operatorname{Cov}(X,Y):=\mathbb{E}\left((X-\mathbb{E}(X))(Y-\mathbb{E}(Y))\right).$$ Since independent variables are also uncorrelated, covariance seems to be some weak form of independence.

Of course there are various nice results for uncorrelated random variables, for example $\mathbb{E}(XY)=\mathbb{E}(X)\mathbb{E}(Y)$ or various forms of convergence results like the law of large numbers or the central limit theorem. But what is the essence of covariance, what intuitive notion does it capture? Every example of uncorrelated yet dependent random variables seem very contrived to me. What would a "real-life-example" of two such variables look like?

$\endgroup$
4
  • $\begingroup$ Is the idea of "correlation" not intuitive? $\endgroup$ Nov 17 '20 at 19:37
  • $\begingroup$ @SamHopkins For me at least the difference between "correlation" and "dependence" is not intuitive. $\endgroup$ Nov 17 '20 at 19:42
  • 2
    $\begingroup$ @JannikPitt Random variables can depend on each other in many different ways. Correlation is a special kind of dependence, where one variable taking larger values implies the other takes larger values as well (or smaller, in case of negative correlation) $\endgroup$
    – Wojowu
    Nov 17 '20 at 19:59
  • $\begingroup$ stats.stackexchange.com/questions/18058/… $\endgroup$
    – user76284
    Nov 17 '20 at 23:38
12
$\begingroup$

$\DeclareMathOperator\Cov{Cov}\DeclareMathOperator\Var{Var}$I. "[W]hy the heck one would be interested in the covariance"? At least for two reasons:

  1. The correlation $$\rho_{X,Y}\mathrel{:=}\frac{\Cov(X,Y)}{\sqrt{\Var X}\sqrt{\Var Y}}$$ is the normalized covariance (assuming $\Var X\ne0$ and $\Var Y\ne0$), with the properties that (i) $-1\le\rho_{X,Y}\le1$, (ii) $\rho_{X,Y}=1$ iff $Y=aX+b$ almost surely (a.s.) for some real $a>0$ and some real $b$, and (iii) $\rho_{X,Y}=-1$ iff $Y=aX+b$ a.s. for some real $a<0$ and some real $b$. So, $\rho_{X,Y}$ characterizes the strength of a linear relationship between $X$ and $Y$.

  2. In contrast with the correlation, the covariance has the important bilinearity property: $$\Cov\Big(\sum_{i=1}^m a_iX_i,\sum_{j=1}^n b_jY_j\Big) =\sum_{i=1}^m \sum_{j=1}^n a_ib_j\Cov(X_i,Y_j)$$ for real $a_i,b_j$ (assuming finite second moments of the $X_i$'s and $Y_j$'s).

II. "Every example of uncorrelated yet dependent random variables seem very contrived to me." As was noted, the correlation characterizes the strength of a linear relationship between $X$ and $Y$. Here is an example of random variable (r.v.) $Y$ completely depending on, but uncorrelated with, a r.v. $X$: Let $X$ be any r.v. with $0<E\lvert X\rvert^3<\infty$ whose distribution is symmetric about $0$, and let $Y\mathrel{:=}X^2$. Then $\rho_{X,Y}=0$ and thus "there is no linear relationship" between $X$ and $Y$, but $Y$ is completely determined by $X$.

$\endgroup$
3
$\begingroup$

To add a few additional comments: Full statistical independence of two variables, say X and Y, is actually equivalent to the absence of correlation between f(X) and g(Y) for all functions f and g. And it is in fact enough to restrict the f and g to a `separating class' of functions, for example the sinusoids (for which we would not even need X and Y to have any moments).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.