2
$\begingroup$

I compute well-known "sample Pearson correlation coefficient" of two vectors:

$r(X,Y) = \frac{\sum ^n _{i=1}(X_i - \bar{X})(Y_i - \bar{Y})}{\sqrt{\sum ^n _{i=1}(X_i - \bar{X})^2} \sqrt{\sum ^n _{i=1}(Y_i - \bar{Y})^2}}$ . So far so good.

I need to generalize it as follows. I need to add "weights", $w_i\in[0..1]$, to the formula.
The idea is that smaller values of $w_i$ make affect of $(X_i,Y_i)$ on $r$ smaller.
(Proportionally to value of $w_i$.)
Without $w_i$, every $(X_i,Y_i)$ and $(Y_j,Y_j)$ affec $r$ equaly.
$w_i$ makes then affect $r$ differently, with different "weight".

Would it be right to calculate $r(X,Y,w)$ as
$r(X',Y')$ where $X'_i=w_i(X_i-\bar{X})$, and $Y'_i=w_i(Y_i-\bar{Y})$ ? Or else, what would be the
right way to insert $w_i$ into the formula ?

Improve this question
$\endgroup$
2
  • $\begingroup$ First of all - what do you need the generalization for? What kind of quantity you would like to measure, what properties you would like to have, etc... $\endgroup$
    – Piotr Migdal
    Commented Mar 11, 2011 at 18:14
  • $\begingroup$ I need it to bevhave just like Person. Except that "older" values affect r less than newer values. I give older values smaller weight = smaller "importance". Does it make sense. For example if all $w_i$ are equal then it would become regular Pearson. $\endgroup$
    – Andrei
    Commented Mar 11, 2011 at 18:57

2 Answers 2

Reset to default
2
$\begingroup$

I think that the correct thing to do is treat the values as if they were repeated $w_i$ times. That is, assume first that the weights $w_i$ are non-negative integers, the data actually comes in the form such that the pair $(x_i, y_i)$ appears $w_i$ times, and compute the standard Pearson correlation for such (repeated) data. Doing so gives: $\bar{X}' = \frac{\sum_{i=1}^n w_i x_i}{\sum_{i=1}^n w_i}$ (and similarly for $Y$). And in terms of the original $n$ data points, the formula is: $$ r(X,Y,w) = \frac{\sum_{i=1}^n w_i (X_i-\bar{X}')(Y_i-\bar{Y}')}{\sqrt{\sum_{i=1}^n w_i (X_i - \bar{X}')^2} \sqrt{\sum_{i=1}^n w_i (Y_i - \bar{Y}')^2}} $$

and then you can use it of course for general real weights $w_i$. I don't think this is equivalent to your suggestion.

Improve this answer
$\endgroup$
2
  • $\begingroup$ @Or zuk: that's exactly how -for instance- SPSS handles that weighting; it's implemented as a standard option in each statistical procedure. Actually I know its use of some studies, where the parameters of a distorted sample were compared with that of a population, where the underrepresentated subsamples are weighted higher and the overrepresented samples were weighted lower such that each subsample/group has the same relative size as in the population. The algorithms of SPSS are openly availabe as far as I recall (www.spss.com) but may be you have to go over the new owner instead (IBM) $\endgroup$
    – Gottfried Helms
    Commented Mar 11, 2011 at 19:31
  • $\begingroup$ Yes, should have guessed this is pretty standard - good to know, thanx. I've actually used it lately for a case control genetic study (where the proportion of cases in the sample is different than in the population) $\endgroup$
    – Or Zuk
    Commented Mar 11, 2011 at 19:38
-1
$\begingroup$

Could this item be what you're looking for?

http://www.utdallas.edu/~herve/Abdi-MCC2007-pretty.pdf

Improve this answer
$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .