29
$\begingroup$

Can anyone give an example of a Markov process which is not a strong Markov process? The Markov property and strong Markov property are typically introduced as distinct concepts (for example in Oksendal's book on stochastic analysis), but I've never seen a process which satisfies one but not the other.

Many thanks -Simon

$\endgroup$
  • 1
    $\begingroup$ I did not quite get the first answer(the one using Brownian Motion). If the process starts at x(not equal to 0), the distribution of X(0) is delta(x) and transition kernels are that of brownian motion and if x = 0 then distribution of x(0) is delta(0) and transition kernels according as a constant stochastic process. How do we mix the 2 processes? Sorry if I am missing something silly. $\endgroup$ – user13166 Feb 22 '11 at 12:14
  • 1
    $\begingroup$ @vinay For $x\ne0$ and $t>0$, let $p_t(x,\cdot)$ denote the Gaussian distribution with mean $x$ and variance $t$ and $p_t(0,\cdot)$ denote the Dirac measure at $0$. For every $x$, let $p_0(x,\cdot)$ denote the Dirac measure at $x$. Then, for every bounded measurable $\varphi$, initial distribution $\nu$ and times $0=t_0\le t_1\le \cdots\le t_n$, $E_\nu[\varphi(X(t_0),X(t_1),\ldots,X(t_n))]$ is the integral you know, involving $\varphi$, $\nu$ and the semi-group $(p_t)_{t\ge0}$. QED. In fact, a good way to understand this example is to try to prove that $(p_t)_{t\ge0}$ is indeed a semi-group. $\endgroup$ – Did Feb 22 '11 at 12:34
15
$\begingroup$

Consider the following continuous Markov process X, starting from position x

  1. if x = 0 then Xt = 0 for all times.
  2. if x ≠ 0 then X is a standard Brownian motion starting from x.

This is not strong Markov (look at times at which it hits zero).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This is only a good example if we consider the semigroup version of Markov property. For the version $\mathbb{P}( \cdots | F_{ \le \tau}) = \mathbb{P}( \cdots | X_{\tau}) $, the strong Markov property still holds. $\endgroup$ – Taro NGUYEN Nov 18 '18 at 12:23
  • $\begingroup$ Ok, but by a slight modification, you can let the starting position $x$ be random. Say, $\mathbb{P}(X_0=0)=\mathbb{P}(X_0=1)=1/2$. Then, if $\tau$ is the first time at which $X$ hits zero, we do not have $\mathbb{P}(\cdots\vert F_{\le\tau})=\mathbb{P}(\cdots\vert X_\tau)$ as $X_\tau=0$ generates the trivial sigma-algebra but $F_{\le\tau}$ includes the event $\{X_0=0\}$ which tells us whether $X$ is stuck at 0 or not. $\endgroup$ – George Lowther Dec 9 '18 at 13:39
  • $\begingroup$ Isn't the issue that makes that this process does not satisfy the strong Markov property an issue that makes that it also does not satisfy the 'regular' Markov property? $\endgroup$ – Sextus Empiricus Nov 17 at 8:03
16
$\begingroup$

A standard example is Exercise 6.17 in Michael Sharpe's book General theory of Markov processes. The process stays at zero for an exponential amount of time, then moves to the right at a uniform speed.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Ah, this is quite a simple example. It fails the strong Markov property, but not as badly as Andrey's and my example. This one still satisfies a restricted version of the strong Markov property, where you only look at predictable stopping times. It is "moderate Markov" (books.google.co.uk/…). $\endgroup$ – George Lowther Oct 27 '10 at 17:51
  • 8
    $\begingroup$ I can't resist giving this quote from Kai Lai Chung's "Lectures from Markov Processes to Brownian Motion (1982)" -- It may be difficult for the novice to appreciate the fact that twenty five years ago a formal proof of the strong Markov property was a major event. Who now is interested in an example in which it does not hold? $\endgroup$ – user6096 Oct 28 '10 at 5:02
11
$\begingroup$

Let $X(t) = f(W(t) + \pi)$, where $W(t)$ is a standard Wiener process and $$f(x) = \begin{cases} (x,0), & x\leq 0 \\\ \\\ (\sin x,1-\cos x), & 0 < x < 2\pi \\\ \\\ (x-2\pi,0), & x\geq 2\pi \end{cases} $$ is a map from $\mathbb R$ to $\mathbb R^2$. $X(t)$ is an $\mathbb R^2$-valued Markov process on $\mathbb R_+$ which is not strongly Markovian. See "A Modern Approach to Probability Theory" by Fristedt and Gray (1997, pp. 626–627).

If the time set is discrete, the ordinary Markov property implies the strong Markov property.

| cite | improve this answer | |
$\endgroup$
  • 4
    $\begingroup$ A little complicated to follow, but quite neat. I see it works because the curve given by f intersects itself so, if you stop at an intersection point, you don't know which part of the curve the process is currently moving on. $\endgroup$ – George Lowther Oct 27 '10 at 17:29
  • $\begingroup$ Isn't important that $W_t$ is a Wiener process on $[0,\infty)$? Otherwise it seems that you can't define the familly of measures induced by $f(X_t)$. What is the probability measure on $W(0) = -\pi$ and $W(0) = \pi$? $\endgroup$ – Conrado Costa Nov 9 '15 at 9:17
  • $\begingroup$ I am failing to see how this satisfies the Markov condition. What I am struggling with is the following: If $X(t) = (0,0)$ then the distribution/expectation of $u>t:X(u)$ will be different depending on the past values $s<t:X(s)$. Therefore, the process does not satisfy the Markov condition. $\endgroup$ – Sextus Empiricus Nov 17 at 8:20
5
$\begingroup$

Reflected Brownian motion on the slit domain $D\setminus [0,\infty)\times \{0\}$ where $D$ is the unit disc in the plane. It is Not strong Markov for hitting times on the slit, but it is Markovian.

| cite | improve this answer | |
$\endgroup$
-1
$\begingroup$

Can anyone give an example of a Markov process which is not a strong Markov process?

The Markov property implies the strong Markov property but the other way around is not true.

'Strong' refers to more rules/conditions that define the property. As a consequence it will be a less restrictive situation. The strong condition applies to a wider set of cases.

The Markov property means that conditional on the present value, the future distribution of a stochastic model does not depend on the past values.

The strong Markov property means that conditional on the present value, and conditional on the present state being a stopping time condition, the future distribution of a stochastic model does not depend on the past values.

$\endgroup$
  • $\begingroup$ I am afraid you got the strong Markov property completely wrong... $\endgroup$ – Mateusz Kwaśnicki Nov 17 at 9:06
  • $\begingroup$ @MateuszKwaśnicki I am not surprised that I am wrong, because I see in many places the opposite, but the line of thought that is displayed here is, I believe, not uncommon (at least not among the novice). I will make it community wiki, then anybody will be invited to edit the post explaining further why the reasoning is wrong. This will make it a nice answer for those that stumble upon this question and have the same wrong reasoning, a reasoning which is not explained in the other answers why it is wrong. $\endgroup$ – Sextus Empiricus Nov 17 at 9:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.