The statement that

everything we call random in the universe really just originates from lack of knowledge

is a perfect summary of the *subjective* approach to probability. This is a philosophical framework in which probability distributions are interpreted as representations of people's beliefs about the world.

If probabilities just represent personal beliefs, why do they have to follow rules? One answer, illustrated by the psychological studies mentioned in Steven Landsburg's answer, is that they don't: people can and do have beliefs which break the mathematical rules of probability. In certain situations, however, having beliefs that break the probability rules can lead to consequences you might not like.

For example, suppose you're playing a game where you're presented with a bunch of tickets with statements on them, like "It rained in Boston on October 5, 1904" and "Mexico will win the 2014 World Cup." You're supposed to write a number—a "price"—on each ticket. When you're done, your opponent will point each ticket either towards you or towards herself. If a ticket is pointed towards you, you lose the number of points equal to the price written on the ticket, but you'll gain 100 points if the statement on the ticket turns out to be true. If a ticket is pointed away from you, you gain the number of points equal to the price, but you'll lose 100 points if the statement turns out to be true. If you like gaining points, and you don't like losing points, what prices should you chose? Your answer is a way of expressing your beliefs about the statements on the tickets.

It turns out that if the prices you write down (divided by the fixed payoff of 100) break the rules of probability, your opponent will always be able to arrange the tickets so you're guaranteed to lose points overall. So, if you're afraid your opponent wants to make you lose points, you'd better make sure your beliefs follow the rules of probability!

This kind of argument for why probabilities should follow certain rules, even if they're just personal beliefs, is called a *Dutch Book argument*. Its development started with some mathematicians named Frank Ramsey and Bruno de Finetti, and continues today.

For a more in-depth introduction to the sujective approach to probability, I recommend the book *Subjective Probability: The Real Thing*, by Richard Jeffrey.

Finally, in light of Yuichiro Fujiwara's answer, I can't resist mentioning how the subjective approach to probability extends to quantum physics.

In the betting game I described earlier, I was implicitly assuming assumed that all the statements on the tickets would turn out to be either true or false. In quantum physics, however, there can be statements which are *incompatible* in the sense that, if you test the truth of one, it becomes physically impossible to test the truth of the other. Compatibility is not transitive: it's possible for two statements $A$ and $C$ to be incompatible even if $A$ is compatible with a statement $B$ which is compatible with $C$.

Within each family of mutually compatible statements, the ordinary rules of probability still apply, and you might think these are the only rules you need to worry about. Physicists have discovered, however, that to produce accurate [1] models of the world, they also have to follow some very subtle extra rules that relate the probabilities of incompatible statements!

These extra rules have some interesting consequences. If I only obey the ordinary rules of probability, there's nothing to stop me from being totally certain about the truth or falsehood of every single statement. I can be totally sure, for example, that the soccer ball Mónica Ocampo just kicked will go into the net, but that her team will lose the game anyway. My unshakable confidence may be silly, but it doesn't break the rules of probability! In quantum mechanics, however, the situation is different. The extra rules still allow me to have total certainty within one family of mutually compatible statements, but they'll typically prevent me from having total certainty across multiple families. This suggests that there are physical limits to how much we can know about the universe, which I think is what Yuichiro Fujiwara meant by saying that some things in quantum mechanics are "definitely random" in the way you described.

[1] If probabilities are personal beliefs, what does it mean for them to be "accurate"? Well, it turns out that under certain circumstances, the rules of probability force your beliefs about the experimental frequencies of events to be related to the probabilities you assign to those events. This is called the *weak law of large numbers*, and I'd love to talk about it if this answer weren't already over a page long...

9more comments