# Construction of a Markov process with prescribed local behavior and state-dependent jump distribution

Let

• $$(E,\mathcal E)$$ be a measurable space
• $$\mathcal E_b:=\left\{f:E\to\mathbb R\mid f\text{ is bounded and }\mathcal E\text{-measurable}\right\}$$
• $$(\kappa_t)_{t\ge0}$$ be a Markov semigroup on $$(E,\mathcal E)$$
• $$Q$$ denote the weak generator of $$(\kappa_t)_{t\ge0}$$; i.e. $$\mathcal D(Q):=\left\{f\in\mathcal E_b\mid\forall x\in E:[0,\infty)\ni t\mapsto(\kappa_tf)(x)\text{ is right-differentiable at }0\right\}$$ and $$(Qf)(x):=\left.\frac{\rm d}{{\rm d}t}(\kappa_tf)(x)\right|_{t=0+}\;\;\;\text{for }x\in E\text{ and }f\in\mathcal D(Q)$$
• $$(\Omega,\mathcal A,\operatorname P)$$ be a probability space
• $$(Y_t)_{t\ge0}$$ be an $$(E,\mathcal E)$$-valued time-homogeneous Markov process on $$(\Omega,\mathcal A,\operatorname P)$$ with transition semigroup $$(\kappa_t)_{t\ge0}$$
• $$\alpha$$ be a transition kernel on $$(E,\mathcal E)$$ and $$Af(x):=\int_E ( f(y)-f(x)) \:\alpha(x,{\rm d}y)\;\;\;\text{for all }x\in E\text{ and }f\in\mathcal D(A):=\mathcal E_b$$

Question: How can we construct an $$(E,\mathcal E)$$-valued time-homogeneous Markov process $$(X_t)_{t\ge0}$$ on $$(\Omega,\mathcal A,\operatorname P)$$ with weak generator $$Lf=Qf+Af\;\;\;\text{for all }f\in\mathcal D(L)\subseteq\mathcal D(Q)\cap\mathcal D(A)?\tag1$$

The idea is that the local behavior between jumps of $$(X_t)_{t\ge0}$$ is described by $$(Y_t)_{t\ge0}$$ and, assuming that $$\alpha(x,B)=c(x)\mu(x,B)$$ for all $$x\in E$$ for some $$\mathcal E$$-measurable $$c:E\to[0,\infty)$$ and a Markov kernel $$\mu$$ on $$(E,\mathcal E)$$, the jumps occur at a state-dependent rate $$c$$ and are performed according to the state-depedendent distribution $$\mu$$.

The process should be described by something like $$X_t=\sum_{n\in\mathbb N_0}1_{[\tau_n,\:\tau_{n+1})}(t)Y^{(n)}_{t-\tau_n}\;\;\;\text{for all }t\ge0\tag1,$$ where $$\tau_n$$ is the time of the $$n$$th-jump and the $$Y^{(n)}$$ are independent copies of $$Y$$.

However, how do we need to define the $$\tau_n$$ precisely and how do we see that the weak generator of $$(1)$$ is actually equal to $$L$$?

I'm aware of the following simpler result: If $$(W_n)_{n\in\mathbb N_0}$$ is a time-homogeneous Markov chain on $$(\Omega,\mathcal A,\operatorname P)$$ with transition kernel $$\kappa$$ and $$(N_t)_{t\ge0}$$ is a Poisson process on $$(\Omega,\mathcal A,\operatorname P)$$ with intensity $$r>0$$ and $$W$$ is independent of $$N$$, then $$Z_t:=W_{N_t}\;\;\;\text{for }t\ge0$$ is a time-homogeneous Markov process with transition semigroup $$\left(e^{t(\kappa-r)}\right)_{t\ge0}$$ and generator $$r\left(\kappa-\operatorname{id}_{\mathcal E_b}\right)$$.

In particular, if $$W$$ is a random walk with step distribution $$\alpha^{-1}\nu$$; i.e. $$W_n=\sum_{i=1}^n\xi_i$$ for all $$n\in\mathbb N$$ for some independent identically $$\alpha^{-1}\nu$$-distributed process $$(Z_n)_{n\in\mathbb N}$$ on $$(\Omega,\mathcal A,\operatorname P)$$, then the generator of $$Z$$ is given by $$\mathcal E_b\ni g\mapsto\int g(\;\cdot\;+y)-g\:\nu({\rm d}y).$$

Maybe a similar construction and hence an expression different from $$(1)$$ from which it is easier to derive the desired result is possible in the setting of this question.

The construction given by the OP is almost correct. Here is a slight correction: $$X_t = \sum_{n=0}^{\infty} 1_{[\tau_n,\tau_{n+1})}(t) Y_{t - \tau_n}^{n} \;, \tag{1}$$ where we have introduced

• $$\{\tau_i\}$$ are a sequence of jump times defined via $$\tau_{i+1}=\tau_i+\xi_i$$, $$\tau_0=0$$, and $$\{\xi_i \} \overset{i.i.d.}{\sim} \operatorname{Exp}(1)$$ ; and,
• $$\{Y^{i}\}$$ are independent realizations of $$Y$$ with $$Y_0^i=x$$ if $$i=0$$ and else sample $$Y_0^i \mid (Y^0, \dots, Y^{i-1}, \xi_0, \dots, \xi_i) \sim \alpha(Y_{\tau_i - \tau_{i-1}}^{i-1}, \cdot)$$ .

In other words, $$X_t = \begin{cases} Y^0_t & t < \tau_1 \;, \\ Y^1_{t-\tau_1} & \tau_1 \le t < \tau_2 \;, \\ Y^2_{t-\tau_2} & \tau_2 \le t < \tau_3 \;, \\ \vdots \end{cases}$$

To see that the weak generator of (1) is indeed $$L=Q+A$$, write $$f (X_t) - f(x) = \rm{I} + \rm{II} + \rm{III}$$ where \begin{align*} \rm{I} &:= (f(X_t) - f(x)) 1_{\{t < \tau_1 \}} \;, \\ \rm{II} &:= (f(Y_0^1) - f(Y_{\tau_1}^0)) 1_{\{t \ge \tau_1 \}} \;, \\ \rm{III} &:= (f(Y_{\tau_1}^0) - f(x) + f(X_t) - f(Y_0^1)) 1_{\{t \ge \tau_1 \}} \;. \end{align*} Then \begin{align*} E[\rm{I}] &= e^{-t} ( \kappa_t f(x) - f(x) ) = e^{-t} E \int_0^t Qf (Y_s^0) ds \;, \\ E[{\rm II} \mid \tau_1 = s] &= E[f(Y_0^1) - f(Y_{s}^0)] 1_{\{ t \ge s \}} = E[ A f(Y_s^0) ] 1_{\{ t \ge s \}} \;, \\ E[ \rm{II} ] &= E \int_0^{\infty} E[ {\rm II} \mid \tau_1 = s] e^{-s} ds = E \int_0^t e^{-s} A f(Y_s^0) ds \;. \end{align*} One can similarly show that $$E( \rm{III} )$$ is $$O(t^2)$$ for $$t \in [0,1]$$. Therefore, combining the above and using $$(e^{-s} - e^{-t}) \le (t-s)$$ for $$t \ge s$$, one obtains that for all $$t \in [0,1]$$ $$E[f(X_t)] - f(x) = E\int_0^t (A f(Y_s^0) + Q f(Y_s^0)) ds + O(t^2) \;.$$ While this construction/analysis covers the case of constant jump rates, the case of state-dependent jump rates can be treated similarly as discussed in the comments below.

• Thank you for your answer. Please clarify what you mean when you say that "$\{Y^i\}$ are independent realizations of $Y$ with $Y_0=x$". With respect to which probability measure are the $Y^i$ independent and what is $x$? Do you intend to introduce a family $\operatorname P_x$ of probability measures such that $\{Y^i:i\in\mathbb N_0\}$ is $\operatorname P_x$-indepndent, $\operatorname P[Y^0_0=x]=1$ and the distribution of $Y^i$ under $\operatorname P_x$ is $\alpha(Y_{\tau_i - \tau_{i-1}}^{i-1}, \cdot)$ for all $x\in E$? Jun 6 at 15:46
• Yes, except that I think you meant to say "the distribution of $Y^i_0$ under $P_x$ is $\alpha(Y^{i-1}_{\tau_i - \tau_{I-1}}, \cdot)$". Jun 6 at 16:05
• Yes, sorry, that's what I've meant. It also seems like that we need some kind of independence between $Y^i$ and $\tau_{i-1}$, since I don't understand how you obtain $E[\rm{I}] = e^{-t} ( \kappa_t f(x) - f(x) )$ without that. Jun 6 at 16:06
• BTW, the more general concatenation of Markov processes we have talked about in the other thread is also described in the following paper: google.com/…. It starts in Chapter 11 on p. 55. I'm not 100% sure, but the scenario considered here should be the special case described in chapter 13.1. Jun 6 at 16:13
• The "transfer kernel" should be our $\alpha$, if I'm not missing something. However, I really struggle to understand why the rather complicated construction described in chapter 11.3 is necessary. If all $X^i$ are the same, doesn't this construction somehow mimic the construction in the proof of the Ionescu-Tulcea theorem from which we can infer the existence of independent processes? If you know something about this stuff, it would be great to hear what you can say. Jun 6 at 16:13