1
$\begingroup$

I'm doing some machine learning stuff and I want to take some random samples and determine if a human agrees with the computer. To do this a user just votes up or down on a given item. Then I want to be able to sort by the items with the highest rating. I want to use something more complicated than simply up-down to get good results.

I've looked into the Wilson Interval Score and it seems like a decent solution, but I'm wondering if there are other alternatives.

I'm going to be using C# 4.0 if that matters but for now I'm strictly interested in the math.

Example below:

Lets suppose I have 3 items and multiple people have voted on them according to the table:

Item    Up    Down
1       6       1
2       60      11
3       100     40

In this example I would like Item 3 to be listed first, item 2 second and 3 third. This is a rough approximation of my expectations.

Item 3 has the most responses and highest relative approval. Item 2 has more responses than Item 1 despite having a lower percentage approval.

I'm trying to list the items in terms of some sort of relative metric and algrotithm without using something like percent approval or net score; something more complicated.

$\endgroup$
2

1 Answer 1

2
$\begingroup$

For the question as posed, the binomial model (Wilson Interval Score) is indeed a good approach.

However you should bear in mind that the votes are not independent events (as a binomial model supposes). You could do better by considering who is voting, and possibly also, who is viewing the list. That opens up a more complex range of models.

Collaborative filters are worth a look as a simple personalised ratings model. Beyond that, you get into designing a probability model to describe users and what they like, then fitting it to the data (e.g. by using the Newton-Raphson method to find the maximum likelihood).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.