All Questions
141 questions
6
votes
2
answers
2k
views
Random walk to stay in an interval forever
Consider a random walk on the real time, starting from $0$. But this time assume that we can decide, for each step $i$, a step size $t_i>0$ to the left or the right with equal probabilities.
To ...
- 7,249
1
vote
1
answer
222
views
Uniqueness of invariant measure for equivalent transition probabilities
Suppose $P(x,dy)$ and $Q(x,dy)$ are two Markov transition kernels on a topological space $E$ equipped with Borel $\sigma$-algebra $\mathcal B(E)$. Suppose for every $x \in E$, $P(x,\cdot)$ and $Q(x, \...
- 17k
4
votes
2
answers
255
views
The necessary sufficient condition for recurrence of a Markovian random walk
Suppose $\sigma_{1},\sigma_{2},...$are i.i.d random variables.$S_{0}=0$. Define $S_{n}=S_{0}+\sum_{i=1}^{n}\sigma_{i}$, then ${S_{n}}$ is a Markovian random walk.
I want to figure out the necessary ...
CommunityBot
- 1
2
votes
0
answers
207
views
markov processes and ergodic theory
For an ergodic Markov Chain
$$
\frac{1}{N}\sum_{i=1}^n f(X_i) \rightarrow E_\pi[f]
$$
where $\pi$ is the invariant distribution. I am also dealing with a Markovian process (a state space model to ...
- 169
2
votes
0
answers
166
views
Must rows of a transition matrix be distinct?
Is it true that for all continuous time Markov processes on a countable state space $S$, we have
all rows of the transition matrix $\mathbf{P}_t$ are distinct for all time $t\in[0,\infty)$ ?
This ...
- 165
5
votes
0
answers
95
views
Most visited vertex in a random walk with place dependent drift
Consider the following Markov chain on $\mathbb{Z}$:
$$
P(x,x+1)=1-P(x,x-1)=\frac{1}{2}+e^{-|x|}\cdot \mathbf{1}_{\{x\neq 0\}}
$$
Do there exist constants $c,C>0$ such that
$$
c\cdot P^t(z,z) \...
- 527
2
votes
1
answer
168
views
Random Walk 2D with dependent weights [closed]
I have spent a lot of time trying to solve this problem but have had no luck so far! Any help would be highly appreciated!
Suppose I have a 3x3 grid as shown below.
(3,1) (3,2) (3,3)
(2,1) (2,2) (...
- 1,247
6
votes
1
answer
387
views
Quasi-stationary distribution for a death process
In the paper, Survival in a quasi-death process by van Doorn and Pollett, the quasi-stationary distribution of a transient CTMC is discussed and QSD for a simple death process is derived.
Consider a ...
CommunityBot
- 1
1
vote
0
answers
44
views
Validating a probability density distribution forecast model for a Markov process
Let's say we have a Markov process $X_t$, and we come up with a forecast model that takes some information from outside world and says: "value $X_{t+1}$ has probability density distribution $P_t(x)$". ...
- 3,085
0
votes
0
answers
117
views
Ergodicity property for continuous-time Harris positive Markov process
I have posted this question on there, but got no answer.
The following theorem is Theorem 13.3.3 of Meyn and Tweedie's Markov Chains and Stochastic Stability on page 328:
Theorem 13.3.3. If $\Phi$ ...
CommunityBot
- 1
0
votes
1
answer
408
views
Generating independent random variable from two correlated random variables
Suppose two random variables $X$ and $V$ are given. I am wondering what kind of condition we need to impose on joint distribution of $V$ and $X$ to make sure that there exists a random variable $Z$ ...
CommunityBot
- 1
1
vote
0
answers
101
views
How to fit a stochastic matrix to given data.?
Given a data sequence of noisy observations of a 3-state Markov chain $X$ -- $y_1$,$y_2$,...$y_n$, with two transition matrices $A_1$ and $A_2$ corresponding to different regions (**) in the (unit) ...
3
votes
1
answer
1k
views
Ergodicity of a Markov chain
Hi,
I'd appreciate some help on a Markov chain result I'm trying to show. I believe the following is sufficient for a continuous time Markov chain $(X_t)$ with a countable state space to be ergodic:
...
- 200
6
votes
2
answers
912
views
References for a physicist migrating to stochastic processes
I've studied "Markov Chains" - Norris and "Measure, Integral and Probability" - Capinski, Kopp. Now, I'm looking for a couple of books (or other references) that help me bridging these two topics. ...
CommunityBot
- 1
1
vote
0
answers
111
views
Conditional probabilities in epidemic model
I was contemplating an epidemic model where infection and recovery rates are determined by links. Here node $i$ is infected first and recovers at a rate $\mu_i$. For all other nodes, the recovery is ...
- 519
12
votes
3
answers
4k
views
How to explain "Feller process" to an undergraduate student?
I had to explain in informal terms what a Feller process was, to undergraduate students who understand Markov property, Poisson processes and such. It was easy to define Levy process as generalisation ...
- 1,769
2
votes
1
answer
665
views
Transition probabilities in coupled Markov chains
I know that for a continuous-time Markov chain, the probability of transition from time $0$ to $t$ is given by $P(t)=e^{Q(t)t}$. I have a system of $N$ interdependent continuous-time Markov chains ...
- 1,769
3
votes
0
answers
305
views
Nonlinear Markov process
Consider the following nonlinear $\mathbb{R}$-valued stochastic recursive sequence:
$ X_{n+1} = F(X_n) + W_{n+1}, \quad (W_n)_{n\ge1} \stackrel{ \scriptsize \mathrm{i.i.d.} }{ \sim } \phi. $
How can ...
- 61
2
votes
0
answers
199
views
CLT for a Markov Renewal Process
Suppose $(X,T)=\{(X_n,T_n)\}_{n\geq0}$ is a Markov renewal process, where $X$ is a finite-state, discrete-time Markov chain with state space $\{1,2,...,R\}$. $T$ is the additive component, more ...
- 41
3
votes
4
answers
681
views
Does the variance of a continuous time, time homogeneous, Markov process starting from one point necessarily not decrease?
Let $x_t$ be a zero mean, time homogeneous Markovian process (chiefly look at the case where the value is in $1$ dimension) over time $t$ starting from $x_0=0$. Is it necessary that, in continuous ...
- 10.3k
3
votes
1
answer
2k
views
Hitting time probability in a Random Walk with possibility to die.
A Random Walker can move of one unit to the right with probability $p$, to the left with probability $q$ and it can jump again to the starting point with probability $r$ and die. Naturally $p+q+r=1$. ...
- 2,239
1
vote
0
answers
196
views
The problem of the drunkard in a valley [closed]
We consider a Markov chain on a subset of positive integers S = {0, 1, 2, 3, .......N}, with transition probabilities defined as follows:
The chain jumps only one unit to the left or right.
p(i, j) =...
4
votes
0
answers
264
views
Generalized Markov Processes on CW complexes of dimension > 1
Markov processes have a large variety of applications to physics and chemistry (as well as many other fields). Such processes are formulated on graphs, i.e., CW complexes of dimension one. It is ...
CommunityBot
- 1
0
votes
1
answer
2k
views
Markov Chain: state reduction
Hi I am trying to understand a proof in a paper (written by Isaac Sonin), I don't know if anyone could give me a clarification on the following:
Firstly we have a Markov chain $\{Y_k\}$ with finite ...
- 23
4
votes
0
answers
282
views
Markov operators and existence of ergodic measures
My question refers to the yesterday's question (see here)
of John Learner and goes as follows:
Can we deduce the existence of an ergodic measure if we know that an invariant measure exists, but the ...
CommunityBot
- 1
0
votes
0
answers
151
views
Inequality relating stationary probabilities and transition probabilities
Let $P$ be the transition probability matrix of a aperiodic irreducible DTMC and let $\pi$ be its stationary distribution. I would like to know if there is any literature on types of Markov chains ...
- 1,655
1
vote
1
answer
293
views
Empirical distribution of a collection of iid Markov chains
Suppose we have $N$ independent 2-point Markov chains each having a rate matrix $Q = [-1,1;1,-1]$ and stationary distribution $\pi = [0.5,0.5]$. At time $t=0$, we initiate the chains so that the ...
- 710
2
votes
1
answer
421
views
Extending Wald's equation to two classes of i.d. random variables?
I try to adopt Wald's equation to a slightly more complex problem. In fact, after a full day, I found some solution now, but it has a confusing argument in the middle. Perhaps somebody can help me at ...
CommunityBot
- 1
14
votes
2
answers
2k
views
Markov chains: invariant measures and explosion
The following seems like such an elementary question, but I didn't get anywhere with it.
Suppose you are considering a Markov chain in continuous time which is transient and has an invariant measure (...
- 3,069
2
votes
1
answer
447
views
MCMC with progressive demollification of delta distributions
Edit: I simplified the example to a canonical case for clarity.
Given an integral $\int_{\Omega}{g(\mathbf{x})}$ with a well-posed integrand $g(\mathbf{x})$ defined on some multidimensional space $\...
- 101
4
votes
0
answers
1k
views
The spectrum of a Markov Operator and Invariant Measures
Suppose I have a discrete-time Markov Chain (in an infinite dimensional state space $\Omega$) with Markov operator $P$, a linear operator on the space of bounded measurable functions on $\Omega$. (Or ...
- 509
3
votes
1
answer
335
views
Stochastic processes having Markov kernels
Let $(\Omega_1, \mathcal{F}_1, P_1)$ and $(\Omega_2, \mathcal{F}_2, P_2)$ be probability spaces and suppose $(X_t)$ and $(Y_t)$ are real-valued stochastic processes defined on the respective spaces. ...
- 1,122
2
votes
1
answer
640
views
Reachability for Markov process
Let $X$ be a Markov process (in continuous or discrete time) and define an event
$$
R(T,A) = (\exists t\leq T: X_t \in A).
$$
I have seen in one paper that
$$
\Pr[R(\infty,A)] = \sup\limits_{\tau} \...
- 1,655
2
votes
1
answer
395
views
Probability-one event for Markov chain
Let $X$ be a Markov chain, with countable state space $I$ and transition probability matrix $P$. $X$ is irreducible, but need not be recurrent. Let $S$ be a fixed subset of $I$.
Define a subset $K$ ...
CommunityBot
- 1
11
votes
1
answer
642
views
Random walk origin return monotinicity
Consider a Markov chain on $\mathbb{Z}^d$ with transition kernel $P$ for adjacent vertices (non-diagonal). Essentially this is a $d$ dimensional random walk with the probability of a transition ...
- 17k
2
votes
2
answers
861
views
Spectral gap of a product of Markov processes
For $m \in [N] \equiv \{1,\dots, N\}$, let $Q^{(m)}$ be the generator of a (well-behaved) continuous-time Markov process on a finite state space $[n_m]$. Write $J \equiv (j_1,\dots,j_N) \in \prod_m [...
- 23.2k
3
votes
3
answers
1k
views
Markov random field with continuous index set
Hi
There's Markov random field (MRF) which, by my Wikipedia-based knowledge, is an extension of Markov chain. I'd like to think of it as going from 1D to higher dimensional spaces. Inherent in its ...
- 1,468
2
votes
2
answers
1k
views
Borel-Cantelli Lemma on MCs (absorbing states)
hi, I'm sorry if the question is silly, but I couldn't get my head around it for a while now.
In Markov Chains (MC) proving that a state is either recurrent or transient is through Borel-Cantelli ...
- 5,721
2
votes
2
answers
1k
views
Counterexample Markov process
Let $X$ be a homogeneous Markov process in a continuous time with value in the set $E$. Suppose that for some $T>0,x\in E, A\subset E$ we have
$$
P_x[X_t\in A] = 0
$$
for all $t\in [0,T]$ but
$$
...
CommunityBot
- 1
4
votes
1
answer
782
views
A simple problem in markov chains
I'm trying to understand a 1954 paper of Kubo intitled "Note on the stochastic theory of resonance absorption". The specific problem can be stated mathematically as follows: let $X(t)$ be a random ...
- 5,721
2
votes
1
answer
186
views
scalar diffusions are reversible
It is well known that under mild assumptions a scalar diffusion $dX_t = a(X_t) dt + \sigma(X_t) dW_t$ with invariant probability distribution $\pi$ is reversible. This is indeed not true for ...
- 17k