Random vs Unknown

Question

Is there any distinction at all between a random quantity and an unknown quantity or is it impossible to distinguish?

Example: 5 minutes in the future, I plan to roll a die the the number of the die roll at that point in time is called X. Suppose the number X will happen to be 4 , although there is no way of knowing that at the current time (pretend I didn't say it was going to be 4, or that I retroactively inserted a 4 into the post after I witnessed it, but let's continue this discussion as if it were unknown).

Is X a deterministic constant (in reality equal to 4), which is simply unknown (once again, pretend I didn't say it was 4)... or is it a random variable uniformly distributed from 1 to 6 (discrete)?

Does there exist any situation where we can say that a number is definitely a random variable and definitely not an unknown deterministic constant or vice versa (definitely an unknown deterministic constant and definitely not random)?

If there is no way to distinguish these two things, then is it possible they are in fact descriptions of the same thing i.e. everything we call random in the universe really just originates from lack of knowledge? Does there exist objective randomness which is not a function of the knowledge any specific observer has?

Thanks!

Answer 1

In some decision-theoretic models, one distinguishes between risk, which is quantifiable, and uncertainty, which is not. (The terminology is due to Frank Knight.) Thus the outcome of the roll of a fair die represents a risk; the question of whether the die is fair (absent any knowledge of where it came from) represents uncertainty.

You could argue that this is a hazy and meaningless distinction (after all, there are presumably some odds at which you'd be willing to bet that the die is fair and other odds at which you wouldn't, which allows us to define (quantitatively) the subjective probability that you attach to its fairness). But empirical evidence suggests that the distinction is worth maintaining, because people do behave differently when faced with Knightian uncertainty than with Knightian risk.

For example: Suppose you face an urn with 30 red balls and 60 blue balls. Some of the blue balls (you are given no indication of how many) are marked with X's. None of the red balls are marked with X's. You get to draw one ball from the urn. Would you rather win \$1000 if the ball is red, or win \$1000 if the ball has an X on it? Most people choose the former. Now a separate question: Would you rather win \$1000 if the ball has no X on it, or win \$1000 if the ball is blue? Most people choose the latter. But this pair of preferences is not consistent with any belief about the probability distribution from which the number of X's is drawn. In other words, the existence of any quantifiable distribution is inconsistent with observed behavior.

This might or might not address the question you intend to ask (which might or might not be a question about applied mathematics, i.e. a question about how best to model the way people make decisions). If your question is whether the universe is determinsitic, that's not a math question, applied or otherwise.

Answer 2

Does there exist any situation where we can say that a number is definitely a random variable and definitely not an unknown deterministic constant or vice versa (definitely an unknown deterministic constant and definitely not random)?

In a sense, yes. An example arises in quantum physics. And this distinction greatly extends information theory and takes it to a new world, i.e., quantum information theory (which I hope counts as mathematics).

You may have heard of a "superposition" of quantum states (like how Schrödinger's cat is simultaneously dead and alive). Roughly speaking, this can be explained that you have one state with probability $\alpha_0$ and another with probability $\alpha_1$ until you take a look at it (or until you "measure" it). Here, we write the former state as $\vert 0 \rangle$ and the latter $\vert 1 \rangle$. In quantum mechanics, the superposition of these two states with their associated probabilities is often expressed as

$$\sqrt{\alpha_0}\vert 0 \rangle + \sqrt{\alpha_1}\vert 1 \rangle,$$

where $\alpha_0 + \alpha_1 = 1$. The point is that you get state $\vert i \rangle$ with probability $\sqrt{\alpha_i}^2=\alpha_i$. The sum is $1$ because you get either $\vert 0 \rangle$ or $\vert 1 \rangle$ after measurement. (Actually this is slgihtly wrong because technically a superposition of this kind lies in the pure state space of a two-level quantum mechanical system, which is the Hilbert space $\mathbb{C}^2$. So the coefficients are complex numbers where the sum of their squares is $1$. But we ignore this point for the sake of simplicity.)

You might wonder if these probabilities $\alpha_i$ represent genuine randomness within mother nature or they're simply "unknown." The latter case would imply that quantum mechanics is incomplete in the sense that even a superposition is actually deterministic but we don't know the true law of physics that governs it; there might be hidden variables which, if we understand them, would allow for the perfect prediction of a measurement outcome.

The current consensus in quantum physics is that they're genuinely random (in the sense that they're not deterministic in the way OP described); Bell's theorem says that hidden variables can't explain experimental results. So, getting outcome $i$ with probability $\alpha_i$ when you take a look at a superposition of quantum states is different from getting outcome $i$ with probability $\alpha_i$ when you look inside a box that contains an already thrown die. In a sense, quantum information science actively exploits this distinction to do what would be impossible without it.

---

The following is my original answer I posted before Douglas commented on Bell's theorem. With a little bit of more quantum mechanics (including the precise definition of superpositions), the following quantum state can be directly used for superdense coding, which increases the rate at which information can be sent through a noiseless channel.

Take a maximally entangled pair $\vert \Phi \rangle$ of qubits (such as two particles jointly in a Bell state) and give one to Alice and the other to Bob:

$$\vert \Phi \rangle = \frac{1}{\sqrt{2}}(\vert 0 \rangle_A\otimes\vert 0 \rangle_B + \vert 1 \rangle_A\otimes\vert 1 \rangle_B),$$

where the subscripts $A$ and $B$ mean qubits are possessed by Alice and Bob respectively.

Assume that Bob waits until Alice measures her qubit. Trivially, Alice gets outcome $0$ with probability $\frac{1}{2}$ and outcome $1$ with also probability $\frac{1}{2}$. You could say this is "definitely random."

Now, Bob measures his qubit. If he doesn't know the outcome Alice already got, to his eye, his outcome is definitely random in the sense that it's $0$ with probability $\frac{1}{2}$ and $1$ with probability $\frac{1}{2}$. However, because their quits are maximally entangled, actually Alice can send classical information (such as a binary bit in the form of 0 or 1) to Bob so that he can predict his outcome before he measures it. This is because $\vert \Phi \rangle$ dictates that Bob must get the exact same outcome as Alice's.

So, in this example, Bob's outcome is already decided in the sense that Alice can tell him what he's going to get. But without this classical information, his quantum information is just random in the sense that it's $0$ with probability $\frac{1}{2}$ and $1$ with probability $\frac{1}{2}$. So, this is an "unknown quantity" that is random in your sense. But because Alice measures her qubit first, her outcome is "definitely random" in your sense; no one can predict what classical bit she's going to get after measurement (assuming quantum mechanics is correct).

Answer 3

This is actually a good question which is studied a lot in mathematical modeling in decision theory and social sciences and also have connections to deep foundational issues of probability theory.

On the question: "Is everything we call random in the universe really just originates from lack of knowledge?"

This is a valid consistent way to regard probability, (even by some views when it comes to probabilities arising in quantum physics).

Moreover, I don't know a general formal way to distinguish between randomness which expresses uncertainty and "genuine" randomness. In more specific scenarios or modelings one makes such distinctions: E.g., in cryptography you can formally talk about scenarios where a deterministic quantity is random for an agent with limited computational power.

One good source regarding your question in social science and economics is the book: Itzhak Gilboa, Theory of Decision under Uncertainty. Cambridge University Press, 2009. (Here are some earlier freely available lecture notes by Gilboa on the same topic.)

Another attempt to model the distinction between uncertain and random is via the notion of fuzzy sets.

See also this question on randomness in nature and this answer

Answer 4

In actuarial science, the notion of risk vs. uncertainty is made more concrete with the concepts of process risk and parameter risk. Process risk is "real" randomness, whereas parameter risk reflects uncertainty about the process. There may be a single process, but we may have no access to what it is (which is usually the case), so parameter risk is one way to deal with this.

This is a pretty good model of lots of real world processes. For example, the number of cars that pass under a bridge can be modeled as a Poisson distribution, which has a single parameter $\lambda$ equal to the mean and variance of the process. But while you can estimate $\lambda$, you cannot know it for certain, and it might change over time. It happens that you can give $\lambda$ an uncertainty structure by assuming it is also a random variable. If you assume $\lambda$ has a gamma distribution, the resulting mixed distribution (i.e. by integrating across $\lambda$) is a negative binomial distribution in the parameters of the gamma distribution. The negative binomial distribution has variance that is higher than its mean, unlike the Poisson distribution where these are equal, and so the mixed model incorporates more uncertainty in a concrete way.

You can take the uncertainty further by making the model itself uncertain: this is often called specification risk. This can get very complex, of course. There are Bayesian models that can implement these concepts on a practical level; e.g. the Kalman filter.

Answer 5

The statement that

everything we call random in the universe really just originates from lack of knowledge

is a perfect summary of the subjective approach to probability. This is a philosophical framework in which probability distributions are interpreted as representations of people's beliefs about the world.

If probabilities just represent personal beliefs, why do they have to follow rules? One answer, illustrated by the psychological studies mentioned in Steven Landsburg's answer, is that they don't: people can and do have beliefs which break the mathematical rules of probability. In certain situations, however, having beliefs that break the probability rules can lead to consequences you might not like.

For example, suppose you're playing a game where you're presented with a bunch of tickets with statements on them, like "It rained in Boston on October 5, 1904" and "Mexico will win the 2014 World Cup." You're supposed to write a number—a "price"—on each ticket. When you're done, your opponent will point each ticket either towards you or towards herself. If a ticket is pointed towards you, you lose the number of points equal to the price written on the ticket, but you'll gain 100 points if the statement on the ticket turns out to be true. If a ticket is pointed away from you, you gain the number of points equal to the price, but you'll lose 100 points if the statement turns out to be true. If you like gaining points, and you don't like losing points, what prices should you chose? Your answer is a way of expressing your beliefs about the statements on the tickets.

It turns out that if the prices you write down (divided by the fixed payoff of 100) break the rules of probability, your opponent will always be able to arrange the tickets so you're guaranteed to lose points overall. So, if you're afraid your opponent wants to make you lose points, you'd better make sure your beliefs follow the rules of probability!

This kind of argument for why probabilities should follow certain rules, even if they're just personal beliefs, is called a Dutch Book argument. Its development started with some mathematicians named Frank Ramsey and Bruno de Finetti, and continues today.

For a more in-depth introduction to the sujective approach to probability, I recommend the book Subjective Probability: The Real Thing, by Richard Jeffrey.

---

Finally, in light of Yuichiro Fujiwara's answer, I can't resist mentioning how the subjective approach to probability extends to quantum physics.

In the betting game I described earlier, I was implicitly assuming assumed that all the statements on the tickets would turn out to be either true or false. In quantum physics, however, there can be statements which are incompatible in the sense that, if you test the truth of one, it becomes physically impossible to test the truth of the other. Compatibility is not transitive: it's possible for two statements $A$ and $C$ to be incompatible even if $A$ is compatible with a statement $B$ which is compatible with $C$.

Within each family of mutually compatible statements, the ordinary rules of probability still apply, and you might think these are the only rules you need to worry about. Physicists have discovered, however, that to produce accurate [1] models of the world, they also have to follow some very subtle extra rules that relate the probabilities of incompatible statements!

These extra rules have some interesting consequences. If I only obey the ordinary rules of probability, there's nothing to stop me from being totally certain about the truth or falsehood of every single statement. I can be totally sure, for example, that the soccer ball Mónica Ocampo just kicked will go into the net, but that her team will lose the game anyway. My unshakable confidence may be silly, but it doesn't break the rules of probability! In quantum mechanics, however, the situation is different. The extra rules still allow me to have total certainty within one family of mutually compatible statements, but they'll typically prevent me from having total certainty across multiple families. This suggests that there are physical limits to how much we can know about the universe, which I think is what Yuichiro Fujiwara meant by saying that some things in quantum mechanics are "definitely random" in the way you described.

---

[1] If probabilities are personal beliefs, what does it mean for them to be "accurate"? Well, it turns out that under certain circumstances, the rules of probability force your beliefs about the experimental frequencies of events to be related to the probabilities you assign to those events. This is called the weak law of large numbers, and I'd love to talk about it if this answer weren't already over a page long...

Answer 6

"Bayesianism versus frequentism" is something you could look into.

I draw a sample of 50 people from a population of 2000000000000 trillion and measure their weights and heights and I fit a least squares line; then I toss them back into the population and draw another 50 at random and fit a least squares line again, and the slope is different because that slope is a random variable. By one defintion, something is random if it changes when you replace the first random sample of 50 with another random sample of 50. The correlation between weight and height in that immense population is uncertain but not random in the sense defined above.

That definition makes sense in a lot of contexts considered by statisticians.

Frequentists are those who assign probability distributions only to things that are "random" in that sense. One may say that a biased die has probability $1/5$ of returning a "$1$", when one actually means that it does that $1/5$ of the time.

Bayesians, on the other hand, assign probability distributions to propositions whose truth-value is uncertain. Thus one might say that the probability that there was life on Mars a billion years ago is $1/2$. By frequentist practices, one would not do that, since one cannot say that that is true in half of all instances.

A question arises: Why should the usual rules of mathematical probability apply when probabilities are interpreted as measuring uncertainty about a proposition but cannot be interpreted as relative frequencies? Various justifications have been published. One of those is the topic of Richard T. Cox's book Algebra of Probable Inference. Another is Bruno de Finetti's thought experiments reducing probability to strategies in idealized gambling.

A Bayesian may assign a probability distribution to the uncertain correlation between weight and height in the whole population mentioned above. A frequentist will not do that, but will speak of a probability distribution of the sample correlation, and even of a confidence distribution (but not a probability distribution) of the population correlation. The difference is that the frequentist assigns probability distributions to things that are random and a Bayesian assigns probability distributions to things that are uncertain.

${{}}$

Answer 7

"Random" and "unknown" need to be distinguished for many reasons. Steven and Yuichiro and other pointed very different and important reasons for that; here are my 2 cents from the viewpoint of a software engineer.

When designing algorithms it becomes very important to distinguish randomness (with any distribution) from unknown quantity because these things arise in different context and must be treated completely differently. And sometimes randomness and unknowness come together, but still have to be treated differently.

For example, consider Bayesian networks. While training the network on the initial inputs the AI algorithm would generally consider the structure of the Bayesian network an unknown quantity, while the transition probabilities would be the random quantities. Although the structure and probabilities come together and are related they have to be treated differently during the learning process due to the desired restrictions on the structure. In particular, you often assume apriori that the structure of the belief network is sparse, but you don't make apriori assumtions about random quantities.

Take a look at the book "Causality" by Judea Pearl: it an interesting treatment of random v. unknown in the context of retrieving causality from correlations. This book also has an excellent treatment of conditional independence that I didn't see in other sources.

Answer 8

I can think of one example where the unknown deterministic is used.

When we use sampling to estimate the population mean, we know that 95% of all samples of a given size $n$ give rise to sample means that are within a certain distance of the population mean. We then turn this on its head and say that we are "95% confident" that the population mean is in a calculated confidence interval.

In my experience, the received wisdom here is taken to be that the population mean is a specific number and so really is an unknown deterministic constant because the population mean is either in or out of the given confidence interval, so that it is false to say that there is a 95% probability that the population mean is in the interval.

However I am tempted to see this as convention only and see the random variable as the taking of the $n$ samples and writing that the probability that the population mean is in the "confidence interval" is 95% a priori the taking of the sample.

Answer 9

In "Theory of Probability and Random Processes" by V.V.Tutubalin (it exists, alas, only in Russian: http://padabum.com/d.php?id=10681), the second half of the book is devoted to applications of probability to real world problems. Unlike most textbooks, it contains not an optimistic list of successful applications, but case studies and analysis of arising difficulties. After reading this book you may agree with the author that successful applications of probabilistic models to practice should often be viewed as miracles.

I like the book a lot. Retelling it is not for a MO comment, so let me give you just one quote. After saying that there is no accurate, systematic, and consistent way to apply mathematics to physics and engineering, Tutubalin writes (p.317): This is especially true for applications of probabilistic models due to the fundamental haziness of the notion of statistical ensemble.