54
$\begingroup$

In this MathStackExchange post the question in the title was asked without much outcome, I feel. Edit: As Douglas Zare kindly observes, there is one more answer in MathStackExchange now.

I am not used to basic Probability, and I am trying to prepare a class that I need to teach this year. I feel I am unable to motivate the introduction of random variables. After spending some time speaking about Kolmogoroff's axioms I can explain that they allow to make the following sentence true and meaningful:

The probability that, tossing a coin $N$ times, I get $n\leq N$ tails equals $$\tag{$\ast$}{N \choose n}\cdot\Big(\frac{1}{2}\Big)^N.$$

But now people (i.e. books I can find) introduce the "random variable $X\colon \Omega\to\mathbb{R}$ which takes values $X(\text{tails})=1$ and $X(\text{heads})=0$" and say that it follows the binomial rule. To do this, they need a probability space $\Omega$: but once one has it, one can prove statement $(\ast)$ above. So, what is the usefulness of this $X$ (and of random variables, in general)?

Added: So far my question was admittedly too vague and I try to emend.

Given a discrete random variable $X\colon\Omega\to\mathbb{R}$ taking values $\{x_1,\dots,x_n\}$ I can define $A_k=X^{-1}(\{x_k\})$ for all $1\leq k\leq n$. The study of the random variable becomes then the study of the values $p(A_k)$, $p$ being the probability on $\Omega$. Therefore, it seems to me that we have not gone one step further in the understanding of $\Omega$ (or of the problem modelled by $\Omega$) thanks to the introduction of $X$.

Often I read that there is the possibility of having a family $X_1,\dots,X_n$ of random variables on the same space $\Omega$ and some results (like the CLT) say something about them. But then

  1. I know no example—and would be happy to discover—of a problem truly modelled by this, whereas in most examples that I read there is either a single random variable; or the understanding of $n$ of them requires the understanding of the power $\Omega^n$ of some previously-introduced measure space $\Omega$.
  2. It seems to me (but admit to have no rigourous proof) that given the above $n$ random variables on $\Omega$ there should exist a $\Omega'$, probably much bigger, with a single $X\colon\Omega'\to\mathbb{R}$ "encoding" the same information as $\{X_1,\dots,X_n\}$. In this case, we are back to using "only" indicator functions. I understand that this process breaks down if we want to make $n\to \infty$, but I also suspect that there might be a deeper reason for studying random variables.

All in all, my doubts come from the fact that random variables still look to me as being a poorer object than a measure (or, probably, of a $\sigma$-algebra $\mathcal{F}$ and a measure whose generated $\sigma$-algebra is finer than $\mathcal{F}$, or something like this); though, they are introduced, studied, and look central in the theory. I wonder where I am wrong.

Caveat: For some reason, many people in comments below objected that "throwing random variables away is ridiculous" or that I "should try to come out with something more clever, then, if I think they are not good". That was not my point. I am sure they must be useful, lest all textbooks would not introduce them. But I was unable to understand why: many useful and kind answers below helped much.

$\endgroup$
24
  • 18
    $\begingroup$ This might be a good question for matheducators.stackexchange.com, but it isn't about research level mathematics, so I don't think it can stay here. $\endgroup$ Sep 22, 2016 at 18:15
  • 3
    $\begingroup$ I'm really clueless as to what the actual question is. $\endgroup$ Sep 22, 2016 at 18:17
  • 4
    $\begingroup$ What is your substitute for the expected value or the standard deviation of a random variable, or do you feel those are not important? Do you not plan to cover the LLN and CLT, or do you restrict yourself to the versions for Bernoulli random variables? $\endgroup$ Sep 22, 2016 at 18:36
  • 5
    $\begingroup$ @FilippoAlbertoEdoardo It seems to me there might several quite different questions lurking there. There is the mathematical question of why one may not always just work with the distributions and forget underlying probability spaces. This question is answerable, even though most answers I can think of are outside the scope of a "basic probability" , which can mean a lot of things. There are also didactic questions, which are very different and greatly limit the scope of answers to the first question. $\endgroup$ Sep 22, 2016 at 20:37
  • 7
    $\begingroup$ Before anyone else votes to close, let me note that questions on how to teach advanced undergraduate subjects have generally been considered on-topic here, even though strictly speaking they don't concern mathematical research. They concern things that research mathematicians frequently do, and that people other than research mathematicians do less frequently if at all, and as such they have been tolerated here. $\endgroup$ Sep 28, 2016 at 2:44

16 Answers 16

20
$\begingroup$

One of your concerns is (let me quote from your question)

Often I read that there is the possibility of having a family X1,…,Xn of random variables on the same space. I know no example—and would be happy to discover—of a problem truly modelled by this, whereas in most examples that I read there is either a single random variable

Here is what I do on the first day of my probability class.

The statistical experiment I describe is: Go to the road outside the college building and consider the first car that goes left to right after your arrival. As we do not know/cannot predict which car in the city might be there it is a statistical experiment. The sample space is the set of all cars in your city (or in your country).

Questions:

  1. How many people are in that car?

  2. What is the amount of petrol in the fuel tank at that time?

  3. How many kilometers the car has travelled that day before you noticed?

  4. What is the wavelength of the color of the car? (admittedly artificial)

All these are random variables on the same sample space.

Answer to question 1 might be useful to a person who sells eatables on the roadside? (more passengers means more business)

Answer to question 2 might help decide if it would be profitable to open a petrol-selling shop here.

I ask students to come up with examples of such statistical experiments instead of coin-tossing and dice-throwing ones.

I got this from a bright student:

Go to the library. Observe the first book that is borrowed by a user that day. Sample space is all books of the library.

Random variables are: Number of pages of that book, Price of that book, How many times it has been borrowed earlier.

$\endgroup$
5
  • 2
    $\begingroup$ Wow, great answer. I think this is exactly what I was looking for. Hopefully my class still need to start, I'll definitely borrow you the car-example and your student the library one. Thanks! $\endgroup$ Oct 19, 2016 at 14:18
  • 2
    $\begingroup$ Glad to know this solves your doubt. I know just elementary approach to probability; suffices to teach engineers. You can use my examples. This should also make clear you why random variables are important; often we are not actually interested in the outcome per se; rather a measurement, some numerical attribute of the outcome. So random variables neatly fit into the model of reality. But in my examples, I am not able to give realistically meaningful ones for sum of random variables; amount of fuel + distance covered makes no sense in physics, or business; but ok in mathematics. $\endgroup$ Oct 19, 2016 at 14:40
  • $\begingroup$ Another popular example is the following: the $6n$ dimensional phase space of a mechanical system of $n$ particles makes up a sample space. Typical random variables (functions on the phase space) are the energy (aka Hamiltonian), entropy, free energy etc. $\endgroup$
    – lcv
    Jun 12, 2017 at 8:31
  • 1
    $\begingroup$ This answer is well meant but misses some deeper points. The first: the set of cars in your city is not a mathematical set. (Can you write down a definition of this set in ZFC or whatever foundation you work with? If not, you're misleading students to think that we have some mathematical object $\Omega$.) Most people will find this critique harsh or pedantic, but it's related with the second point. $\endgroup$ Mar 9, 2019 at 10:49
  • 1
    $\begingroup$ (cont.) Second point: although the OP was not precise in the paragraph you quote, it seem that he was wondering why we usually start with a single random variable $X$ (say the color of a car) and suddenly make an infinte sequence $X_1,X_2,\ldots$ of different but iid variables out of it. No book I've seen addresses this subtlety. How would you explain it with your example of the sample space of all cars? $\endgroup$ Mar 9, 2019 at 10:49
40
$\begingroup$

An honest answer should start with the fact that probabilists usually care more about the distributions of random variables than the underlying probability spaces. Terry Tao has a blog post in which he argues that probabilistic concepts are those that are invariant under extending the underlying probability space. A lot of standard probability concepts such as expectations and variances depend only on the distributions of random variables, and in principle, one could state the strong law of large numbers as a result about infinite product measures.

From a didactic point, starting with distributions is odd though. If we are interested in the average height of the population of the Netherlands, we can start with the distribution of heights, but the motivation of the concept requires us to think of this as the height of actual people and making this formal, requires us to reintroduce the sample space of people in the Netherlands.

When it comes to conditioning, we would have to introduce all variables we might want to condition on, by their distribution in a huge joint probability space of distributions. In many applications, the joint distribution will be supported on the graph of a function and we might well treat this function as a random variable to begin with.

On a more advanced level, there are methods of proof that are based on auxiliary underlying probability space. For example, Skorokhod's representation theorem allows us to study weak convergence, something we care about a lot when working with distributions, in terms of almost sure convergence on an auxiliary underlying probability space.

An area which goes well beyond basic probability in which the underlying probability space cannot be dispensed with is the theory of adapted stochastic processes in continuous time. The filtrations representing information are not represented in the distributions of sample paths. There have been some attempts to define a distribution for adapted processes in a way to preserve the relevant information, the most convincing version can be found in the paper Adapted probability distributions by Hoover and Keisler (see also this book.) The resulting notion is very involved and draws on ideas from model theory unfamiliar to most probabilists. In any case, it has not been widely adopted (no pun intended) in the probability literature.

$\endgroup$
2
  • 2
    $\begingroup$ +1 for mentioning the Netherlands $\endgroup$
    – SBF
    Sep 28, 2016 at 9:16
  • 1
    $\begingroup$ And +1 for the reference to Tao's blog. I had previously accepted this answer, but P Vanchinathan's eventually turned out to be more in the spirit of what I was looking for. $\endgroup$ Oct 19, 2016 at 14:19
16
$\begingroup$

Although in principle the sample space, with its $\sigma$-algebra and probability measure, comes first, things are not always so neat in real life. In applications it is often the random variables (some numerical quantities that you are interested in) that are most important, and the sample space is just scaffolding set up to support them. In fact, this is one of the main things that distinguishes probability theory from measure theory. There is a nice discussion of this in D.H. Fremlin, Measure Theory, Volume 2, Ch. 27.

$\endgroup$
2
  • $\begingroup$ This is more or less what I keep on reading, but do you have an example? Bjørn Kjos-Hanssen's one, for instance, is nice although intricate. Also, Fremlin seems to say that "it is preferable often to forget $\Omega$ altogether, but is there a proper mathematical way of doing so? $\endgroup$ Sep 23, 2016 at 8:05
  • 8
    $\begingroup$ Look at the articles in any probability journal, especially an "applied" one. Sample spaces will be mentioned rarely. Everything is about random variables and their distributions. $\endgroup$ Sep 23, 2016 at 15:41
14
$\begingroup$

At a more basic level than the many enlightening answers that this question has already received, it seems to be a meta-mathematical fact that if one wants to study, analyze, and understand a space (by which I'll mean a set having some additional structure), it is extremely advantageous to study the set of functions from that space to a target space such that the functions respect the structures of the spaces. Examples: A random variable is a function from a probability space to the real numbers. A linear character is a homomorphism from a group to some $\text{GL}_n(k)$. A rational function is a map from a variety to $\mathbb{P}^1$ in the category of algebraic varieties. And so on. And after a while, the functions magically become more natural than the original spaces. Of course, the process repeats and one considers spaces consisting of functions and looks at the functions from these functions spaces to other spaces; e.g., differential and integral operators. So it would actually be quite surprising if something like random variables weren't a fundamental tool in probability. (Addendum: The name "random variable" is terribly misleading to students. You'll want to stress that they are neither "random" nor "variables". They're functions.)

$\endgroup$
5
  • 7
    $\begingroup$ Although I agree with the meta-mathematical principle outlined here in general, it's not clear that it fits the actual reasons that probabilists use random variables, nor the way they think about them. As just one example, my experience has been that probabilists DON'T think of random variables as functions. (Rather, they think of them as random variables.) $\endgroup$
    – Tom Church
    Sep 26, 2016 at 3:53
  • 3
    $\begingroup$ I agree with Tom Church. Formally, they're functions, but probabilists think of a random number as a number and a random graph as a graph. It's just that the actual value of the number or the graph isn't fixed; it varies randomly. Hence the term "random variable". $\endgroup$ Sep 26, 2016 at 17:44
  • $\begingroup$ As someone with an algebraic/arithmetic backgroung, though, I must say that I somehow share Joe Sikverman's idea, or that at least it makes me confident that they can be crucial. But your insigt that probabilists normally think of them as variable is highly valuable, it underlines how many of my difficulties really come from a confusion at level of vocabulary: when I read that something is a random and it is defined as a function I start getting perplexed... $\endgroup$ Sep 28, 2016 at 8:18
  • $\begingroup$ Without meaning to undermine the rest of your answer, in re "linear character", I think that the usual useage reserves this term for a homomorphism $G \to \mathrm{GL}_1(k)$ (not $\mathrm{GL}_n(k)$), or else for the composition of a homomorphism $G \to \mathrm{GL}_n(k)$ with the trace map $\mathrm{GL}_n(k) \to k$ (which preserves a lot of information, but not the actual original homomorphism). $\endgroup$
    – LSpice
    Oct 9, 2017 at 0:01
  • 1
    $\begingroup$ @LSpice Sure, you're right, I should have said a linear representation, not character, since the character is, as you say, the trace of the representation. $\endgroup$ Oct 9, 2017 at 0:15
13
$\begingroup$

Let me try to address the vague question of "why random variables." The short answer is that probability theory without random variables is like language without nouns. When I think about probability theory informally, certain quantities naturally arise that I want to give a name to. These are almost always random variables.

For example, if some random process is occurring and I want to analyze how long it will take before $n$ events will occur, then it's natural to ask for the waiting time until the first event, the waiting time between the first and second events, etc. These $n$ waiting times are random variables.

Or suppose I want to understand the trace of a random matrix. The trace is the sum of the diagonal elements. The diagonal elements are random variables. So I immediately know, by linearity of expectation, that the expected value of the trace is the sum of the expected values of the individual diagonal elements. I also suspect that there will often be some kind of central-limit-theorem-thing going on because I'm summing up a bunch of little random quantities.

The structure of the problem is usually best described in terms of random variables, and the main features of that structure will often remain unchanged even if you change the distributions. Of course when you want to do an actual calculation then you'll need to work with the distributions of the random variables.

$\endgroup$
9
  • 2
    $\begingroup$ Thanks, but my point in the question was rather: in order to introduce random variables, we need probability spaces. Now, it seems to me that upon changing the space and/or the probability, it is always possible to rephrase a question about a random variable in terms of the probability on the fundamental space itself. So, to me, it seems that (mathematically) they lie shallower, rather than deeper, in the theory. $\endgroup$ Sep 23, 2016 at 15:54
  • 9
    $\begingroup$ @FilippoAlbertoEdoardo : There was an analogous debate in algebraic number theory. Some resisted adeles and ideles; they argued that you could always rephrase things in terms of ideals (their constituent parts). While this is technically true, it is generally recognized now that the right way to think about the subject is in terms of adeles and ideles. The fact that X can be rephrased in terms of Y does not mean that Y is deeper or more fundamental. The question is, what is the best way to think about the mathematical structure? The answer in probability theory is: random variables. $\endgroup$ Sep 23, 2016 at 17:15
  • 4
    $\begingroup$ I refer again to the waiting time example. The fundamental mathematical quantity of interest is the waiting time. The distribution that the waiting time happens to satisfy is an incidental feature; you can change it around, but the more fundamental fact is that you're waiting around for something to happen. How long you have to wait is a secondary question. $\endgroup$ Sep 23, 2016 at 17:19
  • 8
    $\begingroup$ Let me also emphasize that when probabilists think about a problem, they think in terms of random variables, because that is the natural language for describing probabilistic phenomena in intuitive terms and for thinking about the mathematical structure. Distributions come in primarily when you need to calculate. Don't be misled by the fact that random variables are formally defined in terms of distributions. Formalism is not always a reliable guide to the real conceptual structure. A sphere is conceptually a geometric object, not a set of ordered triples of Dedekind cuts. $\endgroup$ Sep 23, 2016 at 17:35
  • 3
    $\begingroup$ @FilippoAlbertoEdoardo : One more example. When we study random graphs, the natural things to think about are operations on graphs (e.g., adding or deleting randomly chosen edges and/or vertices) rather than operations on probability distributions on the set of graphs. There are no books with the title, Probability distributions on graphs. $\endgroup$ Sep 24, 2016 at 3:16
12
$\begingroup$

Let $\Omega_n$ be the set of equivalence relations on $\{0,1,2,\dots,n-1\}$, each eq. rel. being equally likely. Let $X$ be the number of classes, and $Y$ the size of the largest class.

Note that when $n\ge 4$, $X$ and $Y$ are not deterministic functions of eachother. For instance, the equivalence relation $$01\mid 23$$ has $X=Y=2$, and $$01\mid 2\mid 3$$ has $X=3$, $Y=2$, so $Y$ does not determine $X$.

The number of sample points in $\Omega_n$ is the Bell number $B_n$, which usually is not a perfect power (1,1,2,5,15,52,203,877,$\dots$).

So $X$ and $Y$ form an example of jointly distributed random variables, the understanding of which does not seem to require the understanding of the power of any previously-introduced measure space...


Edit: Or consider sparse random graphs, let $X$ and $Y$ be some quantities associated with social networks such as cohesion or clustering coefficient.

$\endgroup$
2
  • $\begingroup$ Thanks, this seems nice and indeed goes in the direction of my question. I must confess that I hoped for a somehow more "concrete" example, where I mean "coming from true life": many examples in probability theory comes from tossing coins, throwing dice, estimating broken products, etc... Whereas yours, although very nice, seems (to me) more of an ad hoc construction. $\endgroup$ Sep 23, 2016 at 8:04
  • $\begingroup$ @ Bjørn Kjos-Hannsen: looks promising! Can you speculate briefly upon this application? $\endgroup$ Sep 23, 2016 at 13:08
7
$\begingroup$

Suppose you and I play a game where we each choose our strategies from some set $S$. Sometimes we might want to randomize our strategies. We can model this by saying that we each choose an $S$-valued random variable, or equivalently that we each choose a probability distribution on $S$. Most textbooks choose the latter, So far, there's no need for random variables.

Things get a little more complicated if our random choices are correlated with each other. I play strategy C or D depending on whether it's sunny in Rochester; you play C or D depending on whether it's sunny in Buffalo. 80% of the time, we play identically.

We still don't need random variables: The usual formulation is that a correlated equilibrium consists of a probability distribution on $S\times S$ from which neither of us has any incentive to deviate, in the following sense: We both know the distribution, a pair $(s,t)$ is drawn from that distribution; you are told to play $s$ (without being told the value of $t$), I am told to play $t$ (without being told the value of $s$), and we are always (or almost always) both happy to follow those instructions. That's a little clunky but it works.

Now suppose that you've got the option of making your strategy contingent on any of three observables --- the weather in Buffalo, the weather in Montreal and the weather in Toronto. I have a similar set of choices, and my choices are all correlated with your choices in various ways. We can still describe an equilibrium as a probability distribution on $S\times S$ with certain properties, but this gets very clunky indeed. (Try it and you'll see.) If we describe it in terms of random variables, it's simple: We just replace the strategy sets $S$ with the allowable sets of $S$-valued random variables, and ``equilibrium'' just means equilibrium in the new game with the new strategy sets.

I have struggled to write papers on game theory that formulate everything in terms of probability distributions in order to conform to the standard textbook setup --- but have found that those papers become much easier to write, and much easier to read, with the random variable formulation.

$\endgroup$
1
  • 3
    $\begingroup$ I should say that this answer really goes into the direction I was looking for. If I understand correctly, you are basically saying that yes, we could do without random variables in principle but that this is, even from a theoretical point of view, often a bad language. Most probably like playing with group theory being confined to the generators-relations description: sure, any group can be defined in this way and sure, this is sometimes useful or even crucial: but proving Sylow theorem or something like this using only that language is awkward. $\endgroup$ Sep 28, 2016 at 8:27
6
$\begingroup$

This is an answer by analogy, admittedly even more vague than the question.

By Gelfand duality, commutative $C^*$-algebras carry as much information as compact Hausdorff spaces. Why then do we study both? Because we are actually interested in certain entities which can be viewed either as spaces or as algebras.

I believe the same happens with random variables, it is just that the corresponding duality between probability measure spaces and certain von Neumann algebras is less widely studied (I only first became aware of it from Connes noncommutative geometry book).

$\endgroup$
3
  • $\begingroup$ Well, if it is so—and I somehow agree that it sounds very much plausible—I will have troubles in convincing my first-year engineering students of the usefulness of random variables... ;) $\endgroup$ Sep 23, 2016 at 8:06
  • 1
    $\begingroup$ Freshmen in general are more likely baptised by being forcibly submerged into calculus related stuff rather than into algebra, by some reason. If it would be other way around you would worry about convincing them of the usefulness of $\sigma$-algebras, I guess. $\endgroup$ Sep 23, 2016 at 8:28
  • 1
    $\begingroup$ Indeed, in calculus, students are also taught "variables first". One may argue that the variables of calculus are functions on some state space (in simple cases a finite dimensional smooth manifold). But no one teaches manifolds and state spaces in calculus. $\endgroup$ Sep 23, 2016 at 9:25
6
$\begingroup$

I think the problem is you may have the wrong dictionary in mind — the important thing about a random variable is simply that it's a variable.

In the coinflip problem, I have a collection of $\{ \text{heads}, \text{tails} \}$-valued variables $X_i$ (for $i = 1 \ldots N$) express the value of the $i$-th flip. And I can construct other variable expressions out of these, such as the booleans $$X_i = \text{tails}$$ or the subset of $\mathbb{N}$ $$ \{ i \mid X_i = \text{tails} \} $$ or the natural number $$S = \#\{ i \mid X_i = \text{tails} \} $$

All of this makes sense and is the sort of thing you'd do to describe problems, even if you weren't planning on doing probability theory.

The "random" part is that we are working in a setting that lets us measure the boolean-valued variables (which we call "events"), and we will tend to use measures where the identically true boolean variable has measure 1.

In fact, you can even construct a sample space synthetically by defining a sample to be an ultrafilter on the events and building the Stone space". Then, real-valued expressions correspond to continuous real-valued functions on this space. So this really does closely match the intuition of something varying across the sample space.

Measure spaces can give a more manageable approach to sample spaces, though.

(note that you can do the same thing to measures: build a space so that measurable real-valued functions on the original space become continuous real-valued functions on the new one)

$\endgroup$
1
  • $\begingroup$ I agree that I had a misconception in mind and that your point is quite precise. I guess that my biggest problem came from the fact that I was (erroneously, of course) believing that all r.v. encountered normally were boolean, hence I thought it would have been more reasonable to model them simply measure-theoretically. But you underlined a good point with the fact the random variables are variables. Thanks. $\endgroup$ Sep 25, 2016 at 20:27
6
$\begingroup$

Some users suggested that probabilists think of random variables (r.v.'s) as variables or as numbers, rather than functions. This sounds interesting to me, as I have written a number of papers in probability, but hardly ever thought of r.v.'s as variables or numbers.

To me, the usefulness of r.v.'s is mainly in the convenience of notation. It is a bit simpler to write and, I think, to grasp $\mathbb E X$ than $\int_{\mathbb R}x\,\mu(dx)$, where $\mu$ is the distribution of a r.v. $X$.

Similarly, it is simpler to write and to grasp $\mathbb P(X+Y\le s)$ than $\nu(\{(x,y)\in\mathbb R^2\colon x+y\le s\})$, where $\nu$ is the distribution of a pair $(X,Y)$ of r.v.'s. (Of course, $\mathbb P(X+Y\le s)$ is a convenient abbreviation for $\mathbb P(\{\omega\in\Omega\colon X(\omega)+Y(\omega)\le s\})$.)

Also, outside of mathematics, r.v.'s are what usually comes first in mathematical modeling. Say, first one models errors of measurements as random variables and then thinks how to model the (joint) distribution of those random variables!

$\endgroup$
5
$\begingroup$

I believe the answer you may be looking for is that the notion of random variables (that satisfy various properties including linearity of expectation) can be considered as an interface, which in mathematics is captured by an axiomatization of some sort, in exactly the same way we can capture our intuitive notion of natural numbers by the axioms of a discrete ordered semi-ring plus induction. Notice that it is possible for different (even non-isomorphic) implementations to satisfy the same interface, and that is in fact precisely what we intend to achieve by using an interface instead of the implementation.

Why? Just like in programming (from which I have borrowed this terminology), an interface separates the internal structure from the external properties that we are interested in. Taking the example of natural numbers again, note that we do not care whether we use decimal or binary to represent them, so long as the representations obey the rules of arithmetic. Similarly, in the case of random variables, it is necessary to have a model (implementation) of the probability axioms (interface), which measure theory provides, but the interface has always been the goal. In other words, as long as we use some object through its interface alone, its implementation becomes completely irrelevant. Of course there must be at least one implementation otherwise we are just playing with an object that does not exist (such as non-commutative finite fields)...

See this post for more examples. I decided to post this answer because I feel that the issue is not at all restricted to the concept of random variables. With this perspective, it is easy to see that the more you want in your interface, the harder it is to prove/justify the existence of an implementation. It could even be argued that the things mentioned in other answers to require measure theory do not actually need measure theory in a certain sense. This is because measure theory itself was motivated by an interface requiring a $σ$-algebra with an extended-real valuation on it that is non-negative and countably additive and maps the empty set to $0$. So one could say that to capture those things we could simply add further requirements to our probability axioms.

I think what I have said above is vaguely alluded to in Timothy's post, corresponding to his remark that we often care about features that are not affected by the underlying probability distributions. (Not to say the underlying measure theory, or even the logical foundations!) For example, many of us believe that the linearity of expectation and the central limit theorem (suitably stated) have real-world significance, regardless of what we may or may not think about the set-theoretic foundations of measure theory.

$\endgroup$
6
  • $\begingroup$ Useful perspective. Do you have any resources on either: (a) The relationship between programming-style abstraction and abstraction in maths? (a) Probability taught random-variable-first, rather than probability-space-first? $\endgroup$
    – user105106
    Feb 20, 2017 at 9:53
  • $\begingroup$ @TomFitzhenry: (a) There is not a single correspondence between programming-style abstraction and mathematical abstraction, mainly because programming is far more non-formalized than mathematics. In most programming languages it is the programmer's responsibility to make sure their programs function correctly; behavioural guarantees are not even specifiable in most languages, not to say checkable. There are exceptions but rarely used in industrial applications. See softwareengineering.stackexchange.com/a/279362 for some details on the relatively powerful end of the spectrum. $\endgroup$
    – user21820
    Feb 20, 2017 at 12:35
  • $\begingroup$ @TomFitzhenry: (a) In any case, even programming languages with strong type systems rarely support reasoning about correctness, so arguably they do not provide a very good role model, but it is a start; same way Hilbert's attempted axiomatization of Euclidean geometry is useful in making clear just what points and lines are supposed to be like (Note my usage of "like" is akin to the way different models of an axiomatization share some properties but not others.) My view is that the mathematical notion of axiomatization is the right way to think of interfaces! $\endgroup$
    – user21820
    Feb 20, 2017 at 12:46
  • $\begingroup$ @TomFitzhenry: The main issue I wished to bring attention to is that (as this question shows) few people actually think carefully about what properties exactly they want their mathematical objects to have. It is especially useful in pedagogy because students can learn precisely what they are allowed to do. All too often the rules are not clear to them so they just guess and try mimicking the teacher without proper understanding. $\endgroup$
    – user21820
    Feb 20, 2017 at 12:57
  • $\begingroup$ @TomFitzhenry: (b) I recommend teaching probability after laying down precise axiomatization, which is a non-trivial task! One could think of a random variable $X$ as a program that returns an object sampled from some distribution, and then expressions such as "$P( X < 0 )$" and "$E(X)$" can be given specific meaning axiomatically. We also need to be careful with events; one way would be to to extend the logic so that we can reason "In any outcome, if $X < 0$ then $X < 1$." and from that conclude "$P(X < 0) < P(X < 1)$.". This axiomatization is intuitive but not trivial to justify. $\endgroup$
    – user21820
    Feb 20, 2017 at 13:53
4
$\begingroup$

I think if you're doing very simple things, it will always be easier to just work directly with the probability space rather than introduce random variables. But this rapidly gets less true if you want to do something more complicated.

As a very basic example, suppose you want to get estimates for the probability of a sample from the binomial distribution (or a martingale, which is basically the same proof) being far from expectation (Chernoff or Azuma bounds). You certainly don't want to shove the moment generating function into the problem as you define it, because it makes the problem hard to understand: but you want to have access to the moment generating function in the proof; that's a random variable.

For a more serious example, consider the following stochastic process. You start with $G_0$ being the empty graph on $n$ vertices. In each (integer) time step $t\ge 1$, you select a uniform random pair of vertices which are not at distance one or two in $G_{t-1}$, and add this pair to $G_{t-1}$ to get $G_t$. When no such pairs exist, you stop. This is the triangle-free process; it's nice and easy to define. It is an interesting object to study, but to analyse it up to anywhere near the typical stopping time, you need to keep track of a whole bunch of subgraph counts in $G_t$. You can't easily 'see' these by looking at the underlying probability space, not least because you don't really know what it is until you analyse the process. Of course, what these subgraph counts are is a collection of random variables, and the point is that you can analyse their distributions. I don't think you could do this kind of analysis without implicitly using the concept of a random variable, and then you might as well make it explicit. This should answer your (1).

As to (2), it's true more or less trivially by any of several standard encodings of several real numbers as one, but this really is not an interesting construction, because it loses the intuition you're supposed to get from the collection of random variables.

$\endgroup$
2
  • $\begingroup$ I think your examples are really nice, but since I am really new to this theory I cannot understand the second and the first is already at the boundary of my comprehension. Could you speculate a bit more on that? Thank you. $\endgroup$ Sep 23, 2016 at 9:16
  • 1
    $\begingroup$ For the first example - look up the standard proof of Chernoff's inequalities (e.g. on Wikipedia). You want to analyse the random variable $e^X$ in order to find out something about the binomially distributed $X$. For the second - the process should be clear, analysing it is very hard and it shouldn't be obvious why you need these subgraph counts - I just want to promise you do. $\endgroup$
    – user36212
    Sep 24, 2016 at 7:38
4
$\begingroup$

The intuition behind the probability space $\Omega$ is, I think, that it is the state space of "the system" (the existence of $\Omega$ means that we assume that there actually is something that can be called "THE" all encompassing universal state space, and I think that denying such a thing is the philosophical stance of the Baysian, even if he uses it to prove Bayes law). In that intuition a random variable is some value of the state, which is random because we don't know the state of the system. We can, at best, say what the probability is of a set of states. Ideally we have some a priori symmetry principle which says that each state of the (often huge) state space is equally probable (or more generally and technically that there is some natural $\sigma$-algebra and probability measure on the state space). The goal is then to determine the probability of the outcomes of a function depending on that state.

The canonical example, and the birth of probability theory by Fermat, Pascal and Huygens (http://homepages.wmich.edu/~mackey/Teaching/145/probHist.html), is determining the probability of wins and losses in gambling. Here the state space is the set of all possible hands that can be dealt or all possible n-dice outcomes that can be rolled. The random variable is the loss or gain under the rules of the game. The probability for each state is unambiguous and relatively easy to determine whereas the probability of an outcome for the total number of points in a dice rol or the points in a hand of cards, require the enumeration of the number of ways to realise an outcome.

I think that a lot of the more technical development of the subject came from the desire to formalise statistical mechanics and Boltzmann's principle. Here the two typical examples are the computation of magnetisation of a lattice of spins (the Ising model in d-dimensions) and kinetic gas theory both of which can be seen as application of Boltzmann's principle, which says that without extra information the probability to be in a state x is proportional to $\exp(-\beta E(x))$ for some inverse temperature $\beta = 1/T > 0$. $T$ is called the temperature and equals the thermodynamic temperature in physical systems.

Since the question was about finite systems we only consider the Ising model. For the Ising model in d-dimensions we have a finite state space which is $\Omega = \{-1,1\}^\Lambda$, where $\Lambda = {\{0,1, 2,,...N\}^d}$, i.e. a "spin" $\omega(\lambda)$ with value $±1$ at each integral point $\lambda = (m_1, m_2, m_3)$ of a cubic lattice (part) $\Lambda$ with integral coordinates with $0 \le m_i \le N$ and $N \gg 0$ (in fact $N^3 \approx 10^{23}$). Then the energy of the configuration $E(\omega) = \sum_{\lambda, \mu \in \Lambda, |\lambda -\mu| = 1} \omega(\lambda)\omega(\mu)$. According to the Boltzmann principle, The probability of a state $\omega$ is given by $P(\omega) = exp(-\beta E(\omega))/Z(\beta) $ where $Z(\beta)$ is a normalisation constant called the partition function. The magnetisation is then $M(\omega) = \sum_{\lambda \in \Lambda} \omega(\lambda) / N^3$. The name of the game is then, to determine the expectation value $\mathbb{E} M$ in the limit $N\to \infty$. In $d =2$ this has been done by Ising. Trying to make sense of a state space $\sigma$-algebra and measure in the $N\to \infty$ limit leads you directly to rather serious measure theory and the probability theory of Gibbs-measures (https://www.math.uni-bielefeld.de/~preston/rest/gibbs/files/specifications.pdf)

$\endgroup$
3
$\begingroup$

Random variables are needed because probability distributions are insufficient to describe realistic random phenomena. Indeed, in practical problems we often only have realizations of random variables to work with, and rarely have a formula for their probability distribution. Instead we often work with empirical measures, i.e., measure-valued random elements. In sum, while probability distributions are mathematically appealing, they can only go so far in helping us understand realistic random phenomena.

$\endgroup$
3
$\begingroup$

Practically everything we measure in real life (for instance the time taken for an apple to fall on Newton's head) is "random" in the sense that if we perform the experiment again, we will not get the same answer. So every measurement is a random variable $X$ whose probability of being within $x$ and $x+dx$ is usually $f(x)dx$ where $f(x)$ is the probability density function (or perhaps even better, the probability of $X$ being at least as large as $x$ is the cumulative distribution function).

$\endgroup$
2
$\begingroup$

I think any answer should not be mathematical - as there should not be a mathematical answer to "Why do we need sets/functions/numbers…?". My take is, that random variables are just there. There is no need to need them…

Let's get no too philosophical, but there are things in nature that just appear random or at least with a random component.Some examples, like rolling a dice, do not really need random variables to describe them, because one can really model the whole experiment and talk about events and such. But other examples are not like that: The temperature next Sunday is not totally random, but surely nobody can predict it from now, so let's assume that it is a random number. If there is an underlying probability space is not really important, since all interesting properties of this random number should be independent of it, e.g. that the temperature is above (or below) some value. In other words: the distribution of the random number really matters. So to me it seems, that random variables are just the right way to think about random numbers.

$\endgroup$
1
  • $\begingroup$ Well, although I can agree with your philosophy, my question was more mathematical (somehow): when you say "the underlying probability space is not really important"—you'd agree that mathematically this does not work. Of course, one cares about the temperature, but the underlying theory is needed to put everything on solid ground. And there I got puzzled, since one first needs the probability space and then cans speak about random variables... $\endgroup$ Sep 23, 2016 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.