MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Say I have 5 vectors and I measure the similarity of each one to a fixed reference vector using cosine similarity. But now what I want to do is understand the aggregate or collective strength of these 5 vectors as it relates to how similar they are collectively to the reference vector. Is this simply an average of the 5 cossim values I calculate; or do I add up each scalar position of the 5 vectors together and then do a cossim; or do I average the scalar positions of the 5 vectors and then do a cossim. Or is there a better way to measure the collective similarity (or strength) of the 5 vectors to the reference vector? Thanks for any insight that you may have !

Additional Context So lets say that each of the 5 vectors represent 5 sentences in a document that are considered the most salient bits of information in the document on a topic like say Pneumonia. The question I am asking of the 5 vectors (sentences) is how likely or strong do these 5 sentences represent a positive Pneumonia case? I don't want to consider all sentences in the document because typically these documents are super noisy. What I want is a relative strength or conviction of the 5 vectors (sentences) to represent a Pneumonia case. But how do I determine their aggregate strength? I can measure each vector (sentence) relative to a reference vector for Pneumonia but how do I aggregate the separate results?

Or maybe can I somehow quantify the cluster density around the reference vector?

share|cite|improve this question

I think the answer depends on why you're computing this number. One way to think about it is to look at some toy 2D examples.

For example, suppose your reference vector is (1,0) and you have two comparison vectors, (1,1) and (1,-1). If you average their coordinates, you get (1,0) which is identical to the reference vector--the most "similar" possible; if you average their cosine similarities, you get a less "similar" result. Which would you prefer?

Some notes on what you've said already: if you're only interested in angles (or their cosines) then averaging the reference vectors will give the same result as just adding them. But the fact that you refer to the "collective strength" makes me wonder if you're interested in magnitude as well, in which case you might consider a dot product.

In any case, feel free to add more detail about your specific context, this will help people answer the question.

share|cite|improve this answer
Hi Martin, I am new to this website so I am not sure how I let you know that I added more context as your suggested. Just was wondering if that helped any.... – JPF Oct 29 '09 at 0:38
Thanks, this context is very useful. First, when I've seen people do things like this, they've often just added all the comparison vectors and then computed cosine similarity to the reference. That said, you're really operating in the realm of pragmatic statistics and not math: if you have a set of labeled documents (pneumonia vs. not-pneumonia) you can test various methods. And there are lots of candidates; this is a well-studied area. The Wikipedia page on "Document Classification" has a few pointers. Sorry I can't give a specific answer, but it really depends on your data. – Martin M. W. Oct 29 '09 at 1:08

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.