Why do we need random variables?

Question

In this MathStackExchange post the question in the title was asked without much outcome, I feel. Edit: As Douglas Zare kindly observes, there is one more answer in MathStackExchange now.

I am not used to basic Probability, and I am trying to prepare a class that I need to teach this year. I feel I am unable to motivate the introduction of random variables. After spending some time speaking about Kolmogoroff's axioms I can explain that they allow to make the following sentence true and meaningful:

The probability that, tossing a coin $N$ times, I get $n\leq N$ tails equals $$\tag{$\ast$}{N \choose n}\cdot\Big(\frac{1}{2}\Big)^N.$$

But now people (i.e. books I can find) introduce the "random variable $X\colon \Omega\to\mathbb{R}$ which takes values $X(\text{tails})=1$ and $X(\text{heads})=0$" and say that it follows the binomial rule. To do this, they need a probability space $\Omega$: but once one has it, one can prove statement $(\ast)$ above. So, what is the usefulness of this $X$ (and of random variables, in general)?

Added: So far my question was admittedly too vague and I try to emend.

Given a discrete random variable $X\colon\Omega\to\mathbb{R}$ taking values $\{x_1,\dots,x_n\}$ I can define $A_k=X^{-1}(\{x_k\})$ for all $1\leq k\leq n$. The study of the random variable becomes then the study of the values $p(A_k)$, $p$ being the probability on $\Omega$. Therefore, it seems to me that we have not gone one step further in the understanding of $\Omega$ (or of the problem modelled by $\Omega$) thanks to the introduction of $X$.

Often I read that there is the possibility of having a family $X_1,\dots,X_n$ of random variables on the same space $\Omega$ and some results (like the CLT) say something about them. But then

1. I know no example—and would be happy to discover—of a problem truly modelled by this, whereas in most examples that I read there is either a single random variable; or the understanding of $n$ of them requires the understanding of the power $\Omega^n$ of some previously-introduced measure space $\Omega$.
2. It seems to me (but admit to have no rigourous proof) that given the above $n$ random variables on $\Omega$ there should exist a $\Omega'$, probably much bigger, with a single $X\colon\Omega'\to\mathbb{R}$ "encoding" the same information as $\{X_1,\dots,X_n\}$. In this case, we are back to using "only" indicator functions. I understand that this process breaks down if we want to make $n\to \infty$, but I also suspect that there might be a deeper reason for studying random variables.

All in all, my doubts come from the fact that random variables still look to me as being a poorer object than a measure (or, probably, of a $\sigma$-algebra $\mathcal{F}$ and a measure whose generated $\sigma$-algebra is finer than $\mathcal{F}$, or something like this); though, they are introduced, studied, and look central in the theory. I wonder where I am wrong.

Caveat: For some reason, many people in comments below objected that "throwing random variables away is ridiculous" or that I "should try to come out with something more clever, then, if I think they are not good". That was not my point. I am sure they must be useful, lest all textbooks would not introduce them. But I was unable to understand why: many useful and kind answers below helped much.

Answer 1

One of your concerns is (let me quote from your question)

Often I read that there is the possibility of having a family X1,…,Xn of random variables on the same space. I know no example—and would be happy to discover—of a problem truly modelled by this, whereas in most examples that I read there is either a single random variable

Here is what I do on the first day of my probability class.

The statistical experiment I describe is: Go to the road outside the college building and consider the first car that goes left to right after your arrival. As we do not know/cannot predict which car in the city might be there it is a statistical experiment. The sample space is the set of all cars in your city (or in your country).

Questions:

1. How many people are in that car?

2. What is the amount of petrol in the fuel tank at that time?

3. How many kilometers the car has travelled that day before you noticed?

4. What is the wavelength of the color of the car? (admittedly artificial)

All these are random variables on the same sample space.

Answer to question 1 might be useful to a person who sells eatables on the roadside? (more passengers means more business)

Answer to question 2 might help decide if it would be profitable to open a petrol-selling shop here.

I ask students to come up with examples of such statistical experiments instead of coin-tossing and dice-throwing ones.

I got this from a bright student:

Go to the library. Observe the first book that is borrowed by a user that day. Sample space is all books of the library.

Random variables are: Number of pages of that book, Price of that book, How many times it has been borrowed earlier.

Answer 2

An honest answer should start with the fact that probabilists usually care more about the distributions of random variables than the underlying probability spaces. Terry Tao has a blog post in which he argues that probabilistic concepts are those that are invariant under extending the underlying probability space. A lot of standard probability concepts such as expectations and variances depend only on the distributions of random variables, and in principle, one could state the strong law of large numbers as a result about infinite product measures.

From a didactic point, starting with distributions is odd though. If we are interested in the average height of the population of the Netherlands, we can start with the distribution of heights, but the motivation of the concept requires us to think of this as the height of actual people and making this formal, requires us to reintroduce the sample space of people in the Netherlands.

When it comes to conditioning, we would have to introduce all variables we might want to condition on, by their distribution in a huge joint probability space of distributions. In many applications, the joint distribution will be supported on the graph of a function and we might well treat this function as a random variable to begin with.

On a more advanced level, there are methods of proof that are based on auxiliary underlying probability space. For example, Skorokhod's representation theorem allows us to study weak convergence, something we care about a lot when working with distributions, in terms of almost sure convergence on an auxiliary underlying probability space.

An area which goes well beyond basic probability in which the underlying probability space cannot be dispensed with is the theory of adapted stochastic processes in continuous time. The filtrations representing information are not represented in the distributions of sample paths. There have been some attempts to define a distribution for adapted processes in a way to preserve the relevant information, the most convincing version can be found in the paper Adapted probability distributions by Hoover and Keisler (see also this book.) The resulting notion is very involved and draws on ideas from model theory unfamiliar to most probabilists. In any case, it has not been widely adopted (no pun intended) in the probability literature.

Answer 3

Although in principle the sample space, with its $\sigma$-algebra and probability measure, comes first, things are not always so neat in real life. In applications it is often the random variables (some numerical quantities that you are interested in) that are most important, and the sample space is just scaffolding set up to support them. In fact, this is one of the main things that distinguishes probability theory from measure theory. There is a nice discussion of this in D.H. Fremlin, Measure Theory, Volume 2, Ch. 27.

Answer 4

At a more basic level than the many enlightening answers that this question has already received, it seems to be a meta-mathematical fact that if one wants to study, analyze, and understand a space (by which I'll mean a set having some additional structure), it is extremely advantageous to study the set of functions from that space to a target space such that the functions respect the structures of the spaces. Examples: A random variable is a function from a probability space to the real numbers. A linear character is a homomorphism from a group to some $\text{GL}_n(k)$. A rational function is a map from a variety to $\mathbb{P}^1$ in the category of algebraic varieties. And so on. And after a while, the functions magically become more natural than the original spaces. Of course, the process repeats and one considers spaces consisting of functions and looks at the functions from these functions spaces to other spaces; e.g., differential and integral operators. So it would actually be quite surprising if something like random variables weren't a fundamental tool in probability. (Addendum: The name "random variable" is terribly misleading to students. You'll want to stress that they are neither "random" nor "variables". They're functions.)

Answer 5

Let me try to address the vague question of "why random variables." The short answer is that probability theory without random variables is like language without nouns. When I think about probability theory informally, certain quantities naturally arise that I want to give a name to. These are almost always random variables.

For example, if some random process is occurring and I want to analyze how long it will take before $n$ events will occur, then it's natural to ask for the waiting time until the first event, the waiting time between the first and second events, etc. These $n$ waiting times are random variables.

Or suppose I want to understand the trace of a random matrix. The trace is the sum of the diagonal elements. The diagonal elements are random variables. So I immediately know, by linearity of expectation, that the expected value of the trace is the sum of the expected values of the individual diagonal elements. I also suspect that there will often be some kind of central-limit-theorem-thing going on because I'm summing up a bunch of little random quantities.

The structure of the problem is usually best described in terms of random variables, and the main features of that structure will often remain unchanged even if you change the distributions. Of course when you want to do an actual calculation then you'll need to work with the distributions of the random variables.

Answer 6

Let $\Omega_n$ be the set of equivalence relations on $\{0,1,2,\dots,n-1\}$, each eq. rel. being equally likely. Let $X$ be the number of classes, and $Y$ the size of the largest class.

Note that when $n\ge 4$, $X$ and $Y$ are not deterministic functions of eachother. For instance, the equivalence relation $$01\mid 23$$ has $X=Y=2$, and $$01\mid 2\mid 3$$ has $X=3$, $Y=2$, so $Y$ does not determine $X$.

The number of sample points in $\Omega_n$ is the Bell number $B_n$, which usually is not a perfect power (1,1,2,5,15,52,203,877,$\dots$).

So $X$ and $Y$ form an example of jointly distributed random variables, the understanding of which does not seem to require the understanding of the power of any previously-introduced measure space...

---

Edit: Or consider sparse random graphs, let $X$ and $Y$ be some quantities associated with social networks such as cohesion or clustering coefficient.

Answer 7

Suppose you and I play a game where we each choose our strategies from some set $S$. Sometimes we might want to randomize our strategies. We can model this by saying that we each choose an $S$-valued random variable, or equivalently that we each choose a probability distribution on $S$. Most textbooks choose the latter, So far, there's no need for random variables.

Things get a little more complicated if our random choices are correlated with each other. I play strategy C or D depending on whether it's sunny in Rochester; you play C or D depending on whether it's sunny in Buffalo. 80% of the time, we play identically.

We still don't need random variables: The usual formulation is that a correlated equilibrium consists of a probability distribution on $S\times S$ from which neither of us has any incentive to deviate, in the following sense: We both know the distribution, a pair $(s,t)$ is drawn from that distribution; you are told to play $s$ (without being told the value of $t$), I am told to play $t$ (without being told the value of $s$), and we are always (or almost always) both happy to follow those instructions. That's a little clunky but it works.

Now suppose that you've got the option of making your strategy contingent on any of three observables --- the weather in Buffalo, the weather in Montreal and the weather in Toronto. I have a similar set of choices, and my choices are all correlated with your choices in various ways. We can still describe an equilibrium as a probability distribution on $S\times S$ with certain properties, but this gets very clunky indeed. (Try it and you'll see.) If we describe it in terms of random variables, it's simple: We just replace the strategy sets $S$ with the allowable sets of $S$-valued random variables, and ``equilibrium'' just means equilibrium in the new game with the new strategy sets.

I have struggled to write papers on game theory that formulate everything in terms of probability distributions in order to conform to the standard textbook setup --- but have found that those papers become much easier to write, and much easier to read, with the random variable formulation.

Answer 8

This is an answer by analogy, admittedly even more vague than the question.

By Gelfand duality, commutative $C^*$-algebras carry as much information as compact Hausdorff spaces. Why then do we study both? Because we are actually interested in certain entities which can be viewed either as spaces or as algebras.

I believe the same happens with random variables, it is just that the corresponding duality between probability measure spaces and certain von Neumann algebras is less widely studied (I only first became aware of it from Connes noncommutative geometry book).

Answer 9

I think the problem is you may have the wrong dictionary in mind — the important thing about a random variable is simply that it's a variable.

In the coinflip problem, I have a collection of $\{ \text{heads}, \text{tails} \}$-valued variables $X_i$ (for $i = 1 \ldots N$) express the value of the $i$-th flip. And I can construct other variable expressions out of these, such as the booleans $$X_i = \text{tails}$$ or the subset of $\mathbb{N}$ $$ \{ i \mid X_i = \text{tails} \} $$ or the natural number $$S = \#\{ i \mid X_i = \text{tails} \} $$

All of this makes sense and is the sort of thing you'd do to describe problems, even if you weren't planning on doing probability theory.

The "random" part is that we are working in a setting that lets us measure the boolean-valued variables (which we call "events"), and we will tend to use measures where the identically true boolean variable has measure 1.

In fact, you can even construct a sample space synthetically by defining a sample to be an ultrafilter on the events and building the Stone space". Then, real-valued expressions correspond to continuous real-valued functions on this space. So this really does closely match the intuition of something varying across the sample space.

Measure spaces can give a more manageable approach to sample spaces, though.

(note that you can do the same thing to measures: build a space so that measurable real-valued functions on the original space become continuous real-valued functions on the new one)

Answer 10

Some users suggested that probabilists think of random variables (r.v.'s) as variables or as numbers, rather than functions. This sounds interesting to me, as I have written a number of papers in probability, but hardly ever thought of r.v.'s as variables or numbers.

To me, the usefulness of r.v.'s is mainly in the convenience of notation. It is a bit simpler to write and, I think, to grasp $\mathbb E X$ than $\int_{\mathbb R}x\,\mu(dx)$, where $\mu$ is the distribution of a r.v. $X$.

Similarly, it is simpler to write and to grasp $\mathbb P(X+Y\le s)$ than $\nu(\{(x,y)\in\mathbb R^2\colon x+y\le s\})$, where $\nu$ is the distribution of a pair $(X,Y)$ of r.v.'s. (Of course, $\mathbb P(X+Y\le s)$ is a convenient abbreviation for $\mathbb P(\{\omega\in\Omega\colon X(\omega)+Y(\omega)\le s\})$.)

Also, outside of mathematics, r.v.'s are what usually comes first in mathematical modeling. Say, first one models errors of measurements as random variables and then thinks how to model the (joint) distribution of those random variables!

Answer 11

I believe the answer you may be looking for is that the notion of random variables (that satisfy various properties including linearity of expectation) can be considered as an interface, which in mathematics is captured by an axiomatization of some sort, in exactly the same way we can capture our intuitive notion of natural numbers by the axioms of a discrete ordered semi-ring plus induction. Notice that it is possible for different (even non-isomorphic) implementations to satisfy the same interface, and that is in fact precisely what we intend to achieve by using an interface instead of the implementation.

Why? Just like in programming (from which I have borrowed this terminology), an interface separates the internal structure from the external properties that we are interested in. Taking the example of natural numbers again, note that we do not care whether we use decimal or binary to represent them, so long as the representations obey the rules of arithmetic. Similarly, in the case of random variables, it is necessary to have a model (implementation) of the probability axioms (interface), which measure theory provides, but the interface has always been the goal. In other words, as long as we use some object through its interface alone, its implementation becomes completely irrelevant. Of course there must be at least one implementation otherwise we are just playing with an object that does not exist (such as non-commutative finite fields)...

See this post for more examples. I decided to post this answer because I feel that the issue is not at all restricted to the concept of random variables. With this perspective, it is easy to see that the more you want in your interface, the harder it is to prove/justify the existence of an implementation. It could even be argued that the things mentioned in other answers to require measure theory do not actually need measure theory in a certain sense. This is because measure theory itself was motivated by an interface requiring a $σ$-algebra with an extended-real valuation on it that is non-negative and countably additive and maps the empty set to $0$. So one could say that to capture those things we could simply add further requirements to our probability axioms.

I think what I have said above is vaguely alluded to in Timothy's post, corresponding to his remark that we often care about features that are not affected by the underlying probability distributions. (Not to say the underlying measure theory, or even the logical foundations!) For example, many of us believe that the linearity of expectation and the central limit theorem (suitably stated) have real-world significance, regardless of what we may or may not think about the set-theoretic foundations of measure theory.

Answer 12

I think if you're doing very simple things, it will always be easier to just work directly with the probability space rather than introduce random variables. But this rapidly gets less true if you want to do something more complicated.

As a very basic example, suppose you want to get estimates for the probability of a sample from the binomial distribution (or a martingale, which is basically the same proof) being far from expectation (Chernoff or Azuma bounds). You certainly don't want to shove the moment generating function into the problem as you define it, because it makes the problem hard to understand: but you want to have access to the moment generating function in the proof; that's a random variable.

For a more serious example, consider the following stochastic process. You start with $G_0$ being the empty graph on $n$ vertices. In each (integer) time step $t\ge 1$, you select a uniform random pair of vertices which are not at distance one or two in $G_{t-1}$, and add this pair to $G_{t-1}$ to get $G_t$. When no such pairs exist, you stop. This is the triangle-free process; it's nice and easy to define. It is an interesting object to study, but to analyse it up to anywhere near the typical stopping time, you need to keep track of a whole bunch of subgraph counts in $G_t$. You can't easily 'see' these by looking at the underlying probability space, not least because you don't really know what it is until you analyse the process. Of course, what these subgraph counts are is a collection of random variables, and the point is that you can analyse their distributions. I don't think you could do this kind of analysis without implicitly using the concept of a random variable, and then you might as well make it explicit. This should answer your (1).

As to (2), it's true more or less trivially by any of several standard encodings of several real numbers as one, but this really is not an interesting construction, because it loses the intuition you're supposed to get from the collection of random variables.

Answer 13

The intuition behind the probability space $\Omega$ is, I think, that it is the state space of "the system" (the existence of $\Omega$ means that we assume that there actually is something that can be called "THE" all encompassing universal state space, and I think that denying such a thing is the philosophical stance of the Baysian, even if he uses it to prove Bayes law). In that intuition a random variable is some value of the state, which is random because we don't know the state of the system. We can, at best, say what the probability is of a set of states. Ideally we have some a priori symmetry principle which says that each state of the (often huge) state space is equally probable (or more generally and technically that there is some natural $\sigma$-algebra and probability measure on the state space). The goal is then to determine the probability of the outcomes of a function depending on that state.

The canonical example, and the birth of probability theory by Fermat, Pascal and Huygens (http://homepages.wmich.edu/~mackey/Teaching/145/probHist.html), is determining the probability of wins and losses in gambling. Here the state space is the set of all possible hands that can be dealt or all possible n-dice outcomes that can be rolled. The random variable is the loss or gain under the rules of the game. The probability for each state is unambiguous and relatively easy to determine whereas the probability of an outcome for the total number of points in a dice rol or the points in a hand of cards, require the enumeration of the number of ways to realise an outcome.

I think that a lot of the more technical development of the subject came from the desire to formalise statistical mechanics and Boltzmann's principle. Here the two typical examples are the computation of magnetisation of a lattice of spins (the Ising model in d-dimensions) and kinetic gas theory both of which can be seen as application of Boltzmann's principle, which says that without extra information the probability to be in a state x is proportional to $\exp(-\beta E(x))$ for some inverse temperature $\beta = 1/T > 0$. $T$ is called the temperature and equals the thermodynamic temperature in physical systems.

Since the question was about finite systems we only consider the Ising model. For the Ising model in d-dimensions we have a finite state space which is $\Omega = \{-1,1\}^\Lambda$, where $\Lambda = {\{0,1, 2,,...N\}^d}$, i.e. a "spin" $\omega(\lambda)$ with value $±1$ at each integral point $\lambda = (m_1, m_2, m_3)$ of a cubic lattice (part) $\Lambda$ with integral coordinates with $0 \le m_i \le N$ and $N \gg 0$ (in fact $N^3 \approx 10^{23}$). Then the energy of the configuration $E(\omega) = \sum_{\lambda, \mu \in \Lambda, |\lambda -\mu| = 1} \omega(\lambda)\omega(\mu)$. According to the Boltzmann principle, The probability of a state $\omega$ is given by $P(\omega) = exp(-\beta E(\omega))/Z(\beta) $ where $Z(\beta)$ is a normalisation constant called the partition function. The magnetisation is then $M(\omega) = \sum_{\lambda \in \Lambda} \omega(\lambda) / N^3$. The name of the game is then, to determine the expectation value $\mathbb{E} M$ in the limit $N\to \infty$. In $d =2$ this has been done by Ising. Trying to make sense of a state space $\sigma$-algebra and measure in the $N\to \infty$ limit leads you directly to rather serious measure theory and the probability theory of Gibbs-measures (https://www.math.uni-bielefeld.de/~preston/rest/gibbs/files/specifications.pdf)

Answer 14

Random variables are needed because probability distributions are insufficient to describe realistic random phenomena. Indeed, in practical problems we often only have realizations of random variables to work with, and rarely have a formula for their probability distribution. Instead we often work with empirical measures, i.e., measure-valued random elements. In sum, while probability distributions are mathematically appealing, they can only go so far in helping us understand realistic random phenomena.

Answer 15

Practically everything we measure in real life (for instance the time taken for an apple to fall on Newton's head) is "random" in the sense that if we perform the experiment again, we will not get the same answer. So every measurement is a random variable $X$ whose probability of being within $x$ and $x+dx$ is usually $f(x)dx$ where $f(x)$ is the probability density function (or perhaps even better, the probability of $X$ being at least as large as $x$ is the cumulative distribution function).

Answer 16

I think any answer should not be mathematical - as there should not be a mathematical answer to "Why do we need sets/functions/numbers…?". My take is, that random variables are just there. There is no need to need them…

Let's get no too philosophical, but there are things in nature that just appear random or at least with a random component.Some examples, like rolling a dice, do not really need random variables to describe them, because one can really model the whole experiment and talk about events and such. But other examples are not like that: The temperature next Sunday is not totally random, but surely nobody can predict it from now, so let's assume that it is a random number. If there is an underlying probability space is not really important, since all interesting properties of this random number should be independent of it, e.g. that the temperature is above (or below) some value. In other words: the distribution of the random number really matters. So to me it seems, that random variables are just the right way to think about random numbers.