Skip to main content
7 of 7
added 1 character in body
Nate River
  • 6.3k
  • 2
  • 23
  • 99

A simple stochastic game

An individual, henceforth called the runner starts at the center of an open two dimensional square $\Omega$ of side length $r \geq 2$.

At each turn, a vector $x \in S^1$ is chosen uniformly at random, and the runner may choose to move one unit in the direction $x$ or $-x$ - that is, his new position is just his old position, translated by $x$ or $-x$. The runner’s goal is to stay within $\Omega$ for as long as possible.

Such a choice of moves by the runner is called an admissible strategy. Formally, denoting by $X_i$ the uniform choice of angles at each step, an admissible strategy is a sequence of $\{-1, 1\}$-valued random variables that is adapted to the natural filtration of the $X_i$.

Denote by $P_t$ the runner’s position at time $t$, and given a strategy $S$, define the exit time $\tau_S$ by

$$\tau_S := \inf \{t \in \mathbb Z_+ \, | \,P_t \in \Omega^c\}.$$

It is true, and not overly difficult to prove that given any strategy $S$, $\tau_S$ is finite almost surely.

An optimal strategy is one that maximises the expected time to exit - that is, $S_0$ is optimal if

$$\mathbb E[\tau_{S_0}] = \sup_{S} \mathbb E[\tau_S],$$

where the supremum is taken over all admissible strategies $S$. By compactness there exists at least an optimal strategy.

Question: What is an optimal strategy for the runner to use?

Nate River
  • 6.3k
  • 2
  • 23
  • 99