Extinction of a Markov process with strong drift towards (a neighbourhood of) the absorbing state

Question

Let $(X_t)_{t\in\mathbb{R}\geqslant 0}$ be a Markov Jump Process on a discrete state space $S\cup \{0\}$, with $0$ an absorbing state. If $T_0$ is the hitting time of $0$, I want to prove that

$$ \mathbb{P}_x(T_0<+\infty)=1$$

knowing these facts:

1) For some (finite) neighbourhood of $0$, say $C$, $\mathbb{P}_x(T_C<+\infty)=1\ \ \forall x \notin C$

2) For all $x\in C$, there's a positive probability $p_x$ of eventually reaching $0$,starting from $x$ (This implies there's a uniform probability $p>0$ of reaching $0$ from any state in $C$,as $C$ is finite).

The strategy would be to bound $T_0$ from above, by a Geometric (with parameter $q:=1-p$) sum of iid variables $T_C^{(i)}$ plus the times spent inside $C$.

I would like to write this idea properly. Any help would be appreciated.

Answer 1

Your conditions 1) and 2) imply that the chain is Harris-recurrent in the sense of Definition 6.1.1 in Glynn. By Remark 6.1.1 there, there exists $z\in S$ such that $P_x(\tau(z) < \infty$) = 1 for $x\in S$, where $\tau(z) = \inf\{n\ge1 : X_n = z\}$. From that, your desired conclusion, $P_x(T_0<\infty)=1$, follows immediately. Indeed, $0$ is absorbing and hence $z$ is necessarily $0$, since for any $z\ne0$ one has $P_0(\tau(z)<\infty)=0\ne1$.

However, I do not know how to prove the mentioned remark; perhaps you can ask Peter Glynn about that.

Theorem 14.2.7 in Athreya and Lahiri seems of relevance here, but you seem to lack condition (2.15) in that theorem (which may be somehow established, though).

Answer 2

I see no difference between working with the continuous time process and discrete-time process for this problem. Hence, to mitigate the unnecessary burden of notation, the rest of the work will be presented as the problem is posed for discrete time process.

Denote that:

• $ S'=S\cup \{ 0 \}$

• $C=\{ c_1,c_2,..,c_n\}$

• $ \mathcal{ D}(S'), \mathcal{ D}(C) $ are family of all positive distributions over $S'$ and $C$ , respectively.

• $m(t,V) = \mathbb{P}_{V}( X^t=0 ) $ for $ t \in \mathbb{N}, V \in \mathcal{D}(C)$

• $m(t) = \inf_{ V \in \mathcal{D}(C) } m(t,V)$

• $P$ is the transition matrix of the chain.

Before the demonstration of the problem, we start with some propositions

Firstly with those concerning $m(t)$

Proposition 1: For all $t \in \mathbb{N}$ , $\begin{array}{lccl} m_t : & \mathcal{D}(C) & \rightarrow & \mathbb{R} \\ &V & \mapsto& m(t,V) \end{array} $ is an affine function.

Demonstration 1 If each distribution represented as a stochastic row vector, $m(t,V) $ is indeed equal to $ VP^t\delta_0^T$ .QED

Proposition 2: For each $t$, the infimum in the definition of $m(t)$ is achievable, ie. $ \exists \mu_t \in \mathcal{D}(C) : m(t) = m(t, \mu_t)$.

Demonstration 2 Following from the previous proposition, in looking for the infimum of $m(t,V)$ over $\mathcal{D}(C)$, we are minimizing a continuous function over a compact set ( $ dim( \mathcal{D}(C) ) < \infty $ as $ |C| <\infty$ ). It therefore has a minimum.

Remark 1: As $m_t$ is affine, it is reasonable to even suppose that $\mu_t = \delta_{d_t} $ for some $ d_t \in C$.

Proposition 3: $( m(t) )_{t \in \mathbb{N}}$ is an increasing sequence.

Demonstration 3 The proposition 2 and the fact that $0$ is an absorbing state imply :

$m(t+1) = m(t+1, \mu_{t+1}) = \mathbb{P}_{\mu_{t+1}}( X^{t+1} =0)$

$ \ge \mathbb{P}_{\mu_{t+1}}( X^{t} =0) \ge \min_{\mu \in \mathcal{D}(C)} \mathbb{P}_{\mu}( X^{t} =0) =m(t)$ . QED

Then, some proposition for estimation.

Proposition 4: $ \exists p> 0 $ and $N \in \mathbb{N}$ such that : $ \mathbb{P}_{U}(X^N =0)>p $ for all $U \in \mathcal{D}(C)$ ,ie :

$ m(N) >p$.

Demonstration 4 The second condition of the problem shows that for each $ i \in \mathbb{1,n}$ , there is $p_i>0$ and $N_i>0$ such that :

$\mathbb{P}_{c_i}( X^{N_i} =0 ) > p_i$

Choose $ p= \min( p_1,p_2,..,p_n); N=\max(N_1,N_2,..,N_n)$, using the fact that $0$ is an absorbing state, we deduce that:

$\mathbb{P}_{c}( X^{N} =0 ) > p \forall c \in C$

Therefore, QED.

And the last ingredient,

Proposition 5: $\lim_{ t \rightarrow \infty} m(t) = \inf_{ \mu \in \mathcal{D}(C) } \lim_{t \rightarrow \infty} \mathcal{P}_{\mu}(X^t = 0)= I $

Demonstration 5:

As $0$ is an absorbing state, for all $ \mu \in \mathcal{D}(C) $, $ ( \mathcal{P}_{\mu}(X^t = 0) )_{t \ge 0 }$ is an increasing state.

Therefore, $ \lim_{t \rightarrow \infty} \mathcal{P}_{\mu}(X^t = 0) \ge \mathcal{P}_{\mu}(X^n = 0) \ge m(n) \forall n$

$\Rightarrow RHS \ge LHS$

On the other hand, following the remark 1 , and the fact that $|C| $ is finite, we deduce that :

$\exists$ an strictly increasing sequence of natural numbers $(n_i)_{i \ge 0}$ and a state $c^* \in C$ such that:

$ m(n_i) = m(n_i, \delta_{c^*} ) = \mathbb{P}_{\delta_{c^*} }( X^{n_i} =0)$

As $(m(t))_{t \ge 0}$ is an strictly increasing sequence, we also deduce: $\Rightarrow \lim_{t \rightarrow \infty} m(t) = \lim_{t \rightarrow \infty} \mathbb{P}_{\delta_{c^*} }( X^{t} =0) \ge \inf_{ \mu \in \mathcal{D}(C) } \lim_{t \rightarrow \infty} \mathcal{P}_{\mu}(X^t = 0) $

$\Rightarrow LHS \ge RHS$

QED.

Let's begin the main demonstration

Demonstration

For all $t>0$ , $ \mu \in \mathcal{D}_{C}$ and the fact that $0$ is an absoribng state (*) , we have:

• $ \mathbb{P}_{\mu} (X^{N+t} =0) >p$ ( $N,p$ are defined as in the Proposition 3)

• Take $V \in \mathcal{D}(S' )$ such that $ \mathbb{P}_{\mu} (X^{N} =s |X^N \ne 0 )= \mathbb{P}_{V}( X_0=s)$

  $\Rightarrow \mathbb{P}_{\mu} (X^{N+t} =0 |X^N \ne 0 )= \mathbb{P}_{V}( X^t=0) \forall t $

From those, we deduce:

$ \mathbb{P}_{\mu} (X^{N+t} =0) \ge p+(1-p) \mathbb{P}_{V} (X^{t} =0)$ (1)

On the other hand, we have:

• $ \mathbb{P}_{V} (X^{t} =0) = \sum_{i\ge 0 }^t \mathbb{P}_{V} (X^{t} =0 | T_{C} =i) \mathbb{P}_{V}( T_{C} =i)= E$

  Choose $\mu_i \in \mathcal{D}(C) $ such that:

  • $ \mathbb{P}_{V} (X^{t} =0 | T_{C} =i) =\mathbb{P}_{ \mu_i }( X^{t-i}=0)$( the same way we chose V )

  Therefore,

  $ E= \sum_{i\ge 0 }^t \mathbb{P}_{\mu_i } (X^{t-i} =0 ) \mathbb{P}_{V}( T_{C} =i) \ge \sum_{i\ge 0 }^t m(t-i) \mathbb{P}_{V}( T_{C} =i) $ (2)

From (1) and (2) , we get:

$\mathbb{P}_{\mu} (X^{N+t} =0) \ge p+(1-p)( \sum_{i\ge 0 }^t m(t-i) \mathbb{P}_{V}( T_{C} =i)) $

Take the limit and do some calculations,

$ \lim_{ t \rightarrow \infty} \mathbb{P}_{\mu} (X^{t} =0) \ge p+(1-p)( I ( \sum_{i\ge 0 }^t \mathbb{P}_{V}( T_{C} =i) )$

$ = p+(1-p)I (\mathbb{P}_{V}( T_C<\infty))$

$ = p+(1-p)(I \sum_{s \in S'} \mathbb{P}_{s}( T_C<\infty)\mathbb{P}_V ( X_0=s'))$

$ = p+(1-p)(I \sum_{s \in S'} \mathbb{P}_V ( X_0=s'))$ (first condition)

$ = p+(1-p)I$

Then take the infimum.

$\Rightarrow I \ge p+(1-p)I \Rightarrow I=1$

Therefore, $ \forall \mu \in \mathcal{D}(C) : \lim_{t \rightarrow \infty} \mathbb{P}_{\mu}(X^t = 0)=1$

Which gives $ \mathbb{P}_{\mu}( T_0 < \infty)=1$.()**

Finally, $\forall x \in S'$, we have:

$ \mathbb{P}_{x}( T_0 < \infty) \ge \sum_{i \ge 0 } \mathbb{P}_{x}( T_0 < \infty | T_C = i) \mathbb{P}_{x}( T_C=i) $

$ =^{**} \sum_{i \ge 0 } 1. \mathbb{P}_{x}( T_C=i) = \mathbb{P}_{x}( T_C< \infty)=1$ Q.E.D