Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Problem: Suppose that $K$ different students are ranked based on $N$ different parameters (such as Physics marks, English marks, Biology marks, IQ etc). The rank under each parameter can be repetitive i.e. if two persons have the same IQ then they will have the same rank under the IQ parameter. I want to combine these $N$ individual ranks into a composite rank so we can find find the best overall student to the worst.

Known methods: There are several knows algorithms in literature for for combining individual ranks into a composite ranking. Most of these algorithms work by maximizing a function the $N$ correlations coefficient between the individual ranks under a parameter and the composite rank. Few algorithms maximize the sum of the squares of the individual correlations, other maximize the minima of these $N$ correlation coefficient etc.

I do not find the idea of maximizing a function of the correlation coefficient to create the composite rank to be logically convincing enough to make practical sense although it has statistical backing up. Is there a better way of coming up with a composite rank?

share|improve this question

closed as off topic by Andreas Blass, Steven Landsburg, Goldstern, Andy Putman, Benoît Kloeckner Dec 4 '12 at 18:51

Questions on MathOverflow are expected to relate to research level mathematics within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

There are many different notions of "distance", and I don't see a canonical choice. Different distances will induce different rankings. Are you thinking of a specific distance function? Also, a different distribution of subjects might change the rankings between the students. E.g., if each "math" grade is replaced by an "algebra" and a "geometry" grade, the ranking of students who have bad math grades suddenly worsens. –  Goldstern Dec 4 '12 at 12:49
"I thought the distance was more logically convincing argument than the one where we simply maximized correlation." What "logically convincing argument" did you have in mind? –  Steven Landsburg Dec 4 '12 at 13:24
Your question is ill-posed since there is no "best ranking" until you define what "good" means to you. As pointed out below, the right keywords are "vote theory" and especially "Arrow's theorem". –  Benoît Kloeckner Dec 4 '12 at 18:51

2 Answers 2

Think of the parameters as "voters" and the students as "candidates". Then what you have is an election, and you want to give a set of rules to tally votes and select a winner. However, Arrow's Theorem states that under certain natural conditions your rules will not be compatible with the preferences of all voters.

In other words, the "best student" is not well defined in general (unless the same person is best according to every criterion).

share|improve this answer
This seems unconvincing to me since I see no clear reason you'd want to require independence of irrelevant alternatives here, and plenty of good reason not to. –  Steven Landsburg Dec 4 '12 at 15:51
The bigger problem, of course, is that the original poster hasn't given us a clue what properties he wants his ranking method to have. –  Steven Landsburg Dec 4 '12 at 15:53
The independence of irrelevant alternatives makes a lot of sense to me. A composite ranking that says that student A is the best, but that if we withdraw student B, then C is the best, seems not very convincing to me. I would mitigate the conclusion of Rodrigo Pérez though: the Arrow theorem lead us to choose as ranking one of the given parameter ranking (or rather, given the possibility of draws, to use a lexicographic order). This is much less a problem than for a democratic vote. –  Benoît Kloeckner Dec 4 '12 at 18:50
Benoit: If A is first in math and reading, and third in science, I might want to rank him ahead of B, who is second in all subjects. But if A is first in math and reading, and 2032nd in science, I might want to rank him behind B. This violates IIA. –  Steven Landsburg Dec 4 '12 at 19:37

Maybe your alternative method is improved by considering the real frontier made of the envelope defined from the linear combination of the non-dominated of your points (persons) in the N dimensional space. The performance of each student would then be obtained from the distance not to the optimal theoretical point but to its projection over that real frontier (in fact it should be the ratio between the student point distance to the origin and the distance to the origin of its projection over the frontier). This way you would be ranking against real and comparable performance, and the best student in one discipline could never be considered the worst overall.

This is a ranking in the spirit of DEA (Data Envelopment Analysis), widely covered in the literature.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.