# Construction of a Markov process with prescribed local behavior and state-dependent jump distribution

Let

• $$(E,\mathcal E)$$ be a measurable space
• $$\mathcal E_b:=\left\{f:E\to\mathbb R\mid f\text{ is bounded and }\mathcal E\text{-measurable}\right\}$$
• $$(\kappa_t)_{t\ge0}$$ be a Markov semigroup on $$(E,\mathcal E)$$
• $$Q$$ denote the weak generator of $$(\kappa_t)_{t\ge0}$$; i.e. $$\mathcal D(Q):=\left\{f\in\mathcal E_b\mid\forall x\in E:[0,\infty)\ni t\mapsto(\kappa_tf)(x)\text{ is right-differentiable at }0\right\}$$ and $$(Qf)(x):=\left.\frac{\rm d}{{\rm d}t}(\kappa_tf)(x)\right|_{t=0+}\;\;\;\text{for }x\in E\text{ and }f\in\mathcal D(Q)$$
• $$(\Omega,\mathcal A,\operatorname P)$$ be a probability space
• $$(Y_t)_{t\ge0}$$ be an $$(E,\mathcal E)$$-valued time-homogeneous Markov process on $$(\Omega,\mathcal A,\operatorname P)$$ with transition semigroup $$(\kappa_t)_{t\ge0}$$
• $$\alpha$$ be a transition kernel on $$(E,\mathcal E)$$ and $$Af(x):=\int_E ( f(y)-f(x)) \:\alpha(x,{\rm d}y)\;\;\;\text{for all }x\in E\text{ and }f\in\mathcal D(A):=\mathcal E_b$$

Question: How can we construct an $$(E,\mathcal E)$$-valued time-homogeneous Markov process $$(X_t)_{t\ge0}$$ on $$(\Omega,\mathcal A,\operatorname P)$$ with weak generator $$Lf=Qf+Af\;\;\;\text{for all }f\in\mathcal D(L)\subseteq\mathcal D(Q)\cap\mathcal D(A)?\tag1$$

The idea is that the local behavior between jumps of $$(X_t)_{t\ge0}$$ is described by $$(Y_t)_{t\ge0}$$ and, assuming that $$\alpha(x,B)=c(x)\mu(x,B)$$ for all $$x\in E$$ for some $$\mathcal E$$-measurable $$c:E\to[0,\infty)$$ and a Markov kernel $$\mu$$ on $$(E,\mathcal E)$$, the jumps occur at a state-dependent rate $$c$$ and are performed according to the state-depedendent distribution $$\mu$$.

The process should be described by something like $$X_t=\sum_{n\in\mathbb N_0}1_{[\tau_n,\:\tau_{n+1})}(t)Y^{(n)}_{t-\tau_n}\;\;\;\text{for all }t\ge0\tag1,$$ where $$\tau_n$$ is the time of the $$n$$th-jump and the $$Y^{(n)}$$ are independent copies of $$Y$$.

However, how do we need to define the $$\tau_n$$ precisely and how do we see that the weak generator of $$(1)$$ is actually equal to $$L$$?

I'm aware of the following simpler result: If $$(W_n)_{n\in\mathbb N_0}$$ is a time-homogeneous Markov chain on $$(\Omega,\mathcal A,\operatorname P)$$ with transition kernel $$\kappa$$ and $$(N_t)_{t\ge0}$$ is a Poisson process on $$(\Omega,\mathcal A,\operatorname P)$$ with intensity $$r>0$$ and $$W$$ is independent of $$N$$, then $$Z_t:=W_{N_t}\;\;\;\text{for }t\ge0$$ is a time-homogeneous Markov process with transition semigroup $$\left(e^{t(\kappa-r)}\right)_{t\ge0}$$ and generator $$r\left(\kappa-\operatorname{id}_{\mathcal E_b}\right)$$.

In particular, if $$W$$ is a random walk with step distribution $$\alpha^{-1}\nu$$; i.e. $$W_n=\sum_{i=1}^n\xi_i$$ for all $$n\in\mathbb N$$ for some independent identically $$\alpha^{-1}\nu$$-distributed process $$(Z_n)_{n\in\mathbb N}$$ on $$(\Omega,\mathcal A,\operatorname P)$$, then the generator of $$Z$$ is given by $$\mathcal E_b\ni g\mapsto\int g(\;\cdot\;+y)-g\:\nu({\rm d}y).$$

Maybe a similar construction and hence an expression different from $$(1)$$ from which it is easier to derive the desired result is possible in the setting of this question.

The construction given by the OP is almost correct. Here is a slight correction: $$X_t = \sum_{n=0}^{\infty} 1_{[\tau_n,\tau_{n+1})}(t) Y_{t - \tau_n}^{n} \;, \tag{1}$$ where we have introduced

• $$\{\tau_i\}$$ are a sequence of jump times defined via $$\tau_{i+1}=\tau_i+\xi_i$$, $$\tau_0=0$$, and $$\{\xi_i \} \overset{i.i.d.}{\sim} \operatorname{Exp}(1)$$ ; and,
• $$\{Y^{i}\}$$ are independent realizations of $$Y$$ with $$Y_0^i=x$$ if $$i=0$$ and else sample $$Y_0^i \mid (Y^0, \dots, Y^{i-1}, \xi_0, \dots, \xi_i) \sim \alpha(Y_{\tau_i - \tau_{i-1}}^{i-1}, \cdot)$$ .

In other words, $$X_t = \begin{cases} Y^0_t & t < \tau_1 \;, \\ Y^1_{t-\tau_1} & \tau_1 \le t < \tau_2 \;, \\ Y^2_{t-\tau_2} & \tau_2 \le t < \tau_3 \;, \\ \vdots \end{cases}$$

To see that the weak generator of (1) is indeed $$L=Q+A$$, write $$f (X_t) - f(x) = \rm{I} + \rm{II} + \rm{III}$$ where \begin{align*} \rm{I} &:= (f(X_t) - f(x)) 1_{\{t < \tau_1 \}} \;, \\ \rm{II} &:= (f(Y_0^1) - f(Y_{\tau_1}^0)) 1_{\{t \ge \tau_1 \}} \;, \\ \rm{III} &:= (f(Y_{\tau_1}^0) - f(x) + f(X_t) - f(Y_0^1)) 1_{\{t \ge \tau_1 \}} \;. \end{align*} Then \begin{align*} E[\rm{I}] &= e^{-t} ( \kappa_t f(x) - f(x) ) = e^{-t} E \int_0^t Qf (Y_s^0) ds \;, \\ E[{\rm II} \mid \tau_1 = s] &= E[f(Y_0^1) - f(Y_{s}^0)] 1_{\{ t \ge s \}} = E[ A f(Y_s^0) ] 1_{\{ t \ge s \}} \;, \\ E[ \rm{II} ] &= E \int_0^{\infty} E[ {\rm II} \mid \tau_1 = s] e^{-s} ds = E \int_0^t e^{-s} A f(Y_s^0) ds \;. \end{align*} One can similarly show that $$E( \rm{III} )$$ is $$O(t^2)$$ for $$t \in [0,1]$$. Therefore, combining the above and using $$(e^{-s} - e^{-t}) \le (t-s)$$ for $$t \ge s$$, one obtains that for all $$t \in [0,1]$$ $$E[f(X_t)] - f(x) = E\int_0^t (A f(Y_s^0) + Q f(Y_s^0)) ds + O(t^2) \;.$$ While this construction/analysis covers the case of constant jump rates, the case of state-dependent jump rates can be treated similarly as discussed in the comments below.

• Thank you for your answer. Please clarify what you mean when you say that "$\{Y^i\}$ are independent realizations of $Y$ with $Y_0=x$". With respect to which probability measure are the $Y^i$ independent and what is $x$? Do you intend to introduce a family $\operatorname P_x$ of probability measures such that $\{Y^i:i\in\mathbb N_0\}$ is $\operatorname P_x$-indepndent, $\operatorname P[Y^0_0=x]=1$ and the distribution of $Y^i$ under $\operatorname P_x$ is $\alpha(Y_{\tau_i - \tau_{i-1}}^{i-1}, \cdot)$ for all $x\in E$? Jun 6, 2022 at 15:46
• Yes, except that I think you meant to say "the distribution of $Y^i_0$ under $P_x$ is $\alpha(Y^{i-1}_{\tau_i - \tau_{I-1}}, \cdot)$". Jun 6, 2022 at 16:05
• Yes, sorry, that's what I've meant. It also seems like that we need some kind of independence between $Y^i$ and $\tau_{i-1}$, since I don't understand how you obtain $E[\rm{I}] = e^{-t} ( \kappa_t f(x) - f(x) )$ without that. Jun 6, 2022 at 16:06
• BTW, the more general concatenation of Markov processes we have talked about in the other thread is also described in the following paper: google.com/…. It starts in Chapter 11 on p. 55. I'm not 100% sure, but the scenario considered here should be the special case described in chapter 13.1. Jun 6, 2022 at 16:13
• The "transfer kernel" should be our $\alpha$, if I'm not missing something. However, I really struggle to understand why the rather complicated construction described in chapter 11.3 is necessary. If all $X^i$ are the same, doesn't this construction somehow mimic the construction in the proof of the Ionescu-Tulcea theorem from which we can infer the existence of independent processes? If you know something about this stuff, it would be great to hear what you can say. Jun 6, 2022 at 16:13