33
$\begingroup$

Can anyone give an example of a Markov process which is not a strong Markov process? The Markov property and strong Markov property are typically introduced as distinct concepts (for example in Oksendal's book on stochastic analysis), but I've never seen a process which satisfies one but not the other.

Many thanks -Simon

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I did not quite get the first answer(the one using Brownian Motion). If the process starts at x(not equal to 0), the distribution of X(0) is delta(x) and transition kernels are that of brownian motion and if x = 0 then distribution of x(0) is delta(0) and transition kernels according as a constant stochastic process. How do we mix the 2 processes? Sorry if I am missing something silly. $\endgroup$
    – user13166
    Commented Feb 22, 2011 at 12:14
  • 1
    $\begingroup$ @vinay For $x\ne0$ and $t>0$, let $p_t(x,\cdot)$ denote the Gaussian distribution with mean $x$ and variance $t$ and $p_t(0,\cdot)$ denote the Dirac measure at $0$. For every $x$, let $p_0(x,\cdot)$ denote the Dirac measure at $x$. Then, for every bounded measurable $\varphi$, initial distribution $\nu$ and times $0=t_0\le t_1\le \cdots\le t_n$, $E_\nu[\varphi(X(t_0),X(t_1),\ldots,X(t_n))]$ is the integral you know, involving $\varphi$, $\nu$ and the semi-group $(p_t)_{t\ge0}$. QED. In fact, a good way to understand this example is to try to prove that $(p_t)_{t\ge0}$ is indeed a semi-group. $\endgroup$
    – Did
    Commented Feb 22, 2011 at 12:34

4 Answers 4

Reset to default
18
$\begingroup$

Consider the following continuous Markov process X, starting from position x

  1. if x = 0 then Xt = 0 for all times.
  2. if x ≠ 0 then X is a standard Brownian motion starting from x.

This is not strong Markov (look at times at which it hits zero).

Improve this answer
$\endgroup$
8
  • 1
    $\begingroup$ This is only a good example if we consider the semigroup version of Markov property. For the version $\mathbb{P}( \cdots | F_{ \le \tau}) = \mathbb{P}( \cdots | X_{\tau}) $, the strong Markov property still holds. $\endgroup$
    – Taro Tokyo
    Commented Nov 18, 2018 at 12:23
  • 2
    $\begingroup$ Ok, but by a slight modification, you can let the starting position $x$ be random. Say, $\mathbb{P}(X_0=0)=\mathbb{P}(X_0=1)=1/2$. Then, if $\tau$ is the first time at which $X$ hits zero, we do not have $\mathbb{P}(\cdots\vert F_{\le\tau})=\mathbb{P}(\cdots\vert X_\tau)$ as $X_\tau=0$ generates the trivial sigma-algebra but $F_{\le\tau}$ includes the event $\{X_0=0\}$ which tells us whether $X$ is stuck at 0 or not. $\endgroup$
    – George Lowther
    Commented Dec 9, 2018 at 13:39
  • 1
    $\begingroup$ Isn't the issue that makes that this process does not satisfy the strong Markov property an issue that makes that it also does not satisfy the 'regular' Markov property? $\endgroup$
    – Sextus Empiricus
    Commented Nov 17, 2020 at 8:03
  • $\begingroup$ @SextusEmpiricus I don't think so because if $W$ is a BM then for all $t$ $\mathbb{P}(W_t=0)=0$, so we can say $W_{t+1}|W_t$ is distributed as $N(W_t,1)$ if $W_t\neq 0$ and as the $0$ constant if $W_t=0$. Note that the same transition kernel also works when $Y$ is the $0$ process, and hence it works for any process $X$ constructed by "mixing" $W$ and $Y$. $\endgroup$
    – No-one
    Commented Jul 29, 2023 at 13:10
  • $\begingroup$ @Titti I don't follow how that transition kernel follows from W being a standard Brownian motion. If $W_{t+1}|W_t \sim \text{constant}(0)$ when $W_t = 0$, then don't we have instead a Brownian motion with an absorption state at zero? $\endgroup$
    – Sextus Empiricus
    Commented Jul 29, 2023 at 14:22
18
$\begingroup$

A standard example is Exercise 6.17 in Michael Sharpe's book General theory of Markov processes. The process stays at zero for an exponential amount of time, then moves to the right at a uniform speed.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Ah, this is quite a simple example. It fails the strong Markov property, but not as badly as Andrey's and my example. This one still satisfies a restricted version of the strong Markov property, where you only look at predictable stopping times. It is "moderate Markov" (books.google.co.uk/…). $\endgroup$
    – George Lowther
    Commented Oct 27, 2010 at 17:51
  • 9
    $\begingroup$ I can't resist giving this quote from Kai Lai Chung's "Lectures from Markov Processes to Brownian Motion (1982)" -- It may be difficult for the novice to appreciate the fact that twenty five years ago a formal proof of the strong Markov property was a major event. Who now is interested in an example in which it does not hold? $\endgroup$
    – user6096
    Commented Oct 28, 2010 at 5:02
12
$\begingroup$

Let $X(t) = f(W(t) + \pi)$, where $W(t)$ is a standard Wiener process and $$f(x) = \begin{cases} (x,0), & x\leq 0 \\\ \\\ (\sin x,1-\cos x), & 0 < x < 2\pi \\\ \\\ (x-2\pi,0), & x\geq 2\pi \end{cases} $$ is a map from $\mathbb R$ to $\mathbb R^2$. $X(t)$ is an $\mathbb R^2$-valued Markov process on $\mathbb R_+$ which is not strongly Markovian. See "A Modern Approach to Probability Theory" by Fristedt and Gray (1997, pp. 626–627).

If the time set is discrete, the ordinary Markov property implies the strong Markov property.

Improve this answer
$\endgroup$
3
  • 4
    $\begingroup$ A little complicated to follow, but quite neat. I see it works because the curve given by f intersects itself so, if you stop at an intersection point, you don't know which part of the curve the process is currently moving on. $\endgroup$
    – George Lowther
    Commented Oct 27, 2010 at 17:29
  • $\begingroup$ Isn't important that $W_t$ is a Wiener process on $[0,\infty)$? Otherwise it seems that you can't define the familly of measures induced by $f(X_t)$. What is the probability measure on $W(0) = -\pi$ and $W(0) = \pi$? $\endgroup$
    – Conrado Costa
    Commented Nov 9, 2015 at 9:17
  • $\begingroup$ I am failing to see how this satisfies the Markov condition. What I am struggling with is the following: If $X(t) = (0,0)$ then the distribution/expectation of $u>t:X(u)$ will be different depending on the past values $s<t:X(s)$. Therefore, the process does not satisfy the Markov condition. $\endgroup$
    – Sextus Empiricus
    Commented Nov 17, 2020 at 8:20
6
$\begingroup$

Reflected Brownian motion on the slit domain $D\setminus [0,\infty)\times \{0\}$ where $D$ is the unit disc in the plane. It is Not strong Markov for hitting times on the slit, but it is Markovian.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .