6
$\begingroup$

Given several random variables distributed according to different beta distributions, how can I calculate the probability that any one of those random variables is actually the highest?

The application for this is that I have several ads which people can view and possibly click. Over time I collect more and more data about each ad (views and clicks, which I can use to determine a beta distribution), and I'm interested in picking the ad that people most like to click on.

I need to determine when it is safe to select the ad with the highest click-through rate, so I need to know when a particular ad exceeds a threshold probability of being the best.

ps. I'm a software engineer not a mathematician, I would appreciate it if answers could bear this in mind by avoiding any complex notation ;)

Improve this question
$\endgroup$

2 Answers 2

Reset to default
9
$\begingroup$

The best approach depends on how many beta random variables you are comparing. If you're comparing two beta random variables, numerical integration is the most efficient approach. see this tech report: http://www.bepress.com/mdandersonbiostat/paper46/.

If you're comparing many random variables, simulation will be much easier to implement and possibly more efficient than integration. That is, draw a random sample from each distribution and note which one is largest. Then repeat this process a large number of times. (How many times depends on the accuracy you need. Maybe 1000 would be a good place to start.)

One final suggestion. If your beta parameters are all large, you can use a normal approximation to the beta. How large is "large" depends again how the accuracy you need.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you! Being a programmer, simulation is something I could actually manage quite easily :-) Yes, we're talking about anywhere from 2 to 20 random variables. The typical "click-rate" in my application is 1 click for every 1000 views, but this can vary from 1 in 10,000 to 1 in 100 - depending on the ad. $\endgroup$
    – sanity
    Commented Mar 6, 2012 at 16:05
5
$\begingroup$

For the case of two beta random variables, and the beta parameters alpha and beta both being integers, which I think they should be given that the use case is views versus clicks, there is a closed form analytical solution for the probability that one distribution is greater than the other, from Evan Miller's web article :http://www.evanmiller.org/bayesian-ab-testing.html#cite1.

Specifically eqn 6 of the article gives a closed form solution and there is also some pseudocode included.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .