All Questions
141 questions
1
vote
0
answers
41
views
Asymptotic mixing time and Euclidean probability distance for path graphs
We are given a simple path graph $P(V,E)$ with vertex set $V$ and edge set $E$, having $n=|V|$ nodes. Given an initial distribution $\mathbf{\mu}$ over $V$, let $d_t(\mathbf{\mu},\pi)$ be defined as $\...
- 1,677
2
votes
1
answer
59
views
The ranked mass process associated with a Lambda-coalescent
I am reading a paper by Pitman (1999), and I am confused by his Corollary 7. First some notation so that I can explain my confusion:
$\mathcal{P}_\infty$ is the space of partitions of $\mathbb{N}$, $\...
- 203
5
votes
1
answer
192
views
Non-equivalent definitions of Markov process
As far as I know, there are three definitions of Markov processes (or of Markov chains).
DEFINITION 1 (WEAKER). A process $(X_t)_{t\in[0,\infty)}$ on $(\Omega,\mathcal{F},\mathbb{P})$ with values in ...
- 1,149
1
vote
0
answers
114
views
An urn model with weighted objects and replacement
Consider the following game:
In an urn, there are $K$ balls, $x_0$ of them are blue and light (mass $m_0$), $x_1$ are blue and heavy ($m_1$), $x_2$ are red and light ($m_2$), the rest $x_3$ are red ...
- 31
5
votes
2
answers
369
views
Markov process on a torus with prescribed invariant distribution
In Euclidean space, $\mathbb R^d$, the Langevin diffusion $${\rm d}X_t=b(X_t){\rm d}t+\sigma(X_t){\rm d}W_t\tag1,$$ where $\sigma:\mathbb R^d\to\mathbb R^{d\times k}$, $$b:=\frac{\Sigma+U}2\nabla\ln p+...
- 167
0
votes
0
answers
101
views
Simulation of Markov processes with exponential timestepping
Let $(Y_t)_{t\ge0}$ be a time-homogeneous Markov process with transition semigroup $(\kappa_t)_{t\ge0}$. Numerical simulation of $(Y_t)_{t\ge0}$ can be done in the following way:
Choose an initial ...
- 167
5
votes
0
answers
271
views
How to play golf in one dimension?
One-dimensional golf is a function $g$ on $\mathbb R$ such that
$g(x)= 1+\min_\mu E[g(x+N(\mu,c\mu^2))]$ if $|x|>1$ and 0 if $|x|\le 1.$
Here $N$ is the normal distribution, whose mean $\mu$ you ...
- 18.8k
2
votes
0
answers
54
views
Including fixed-time transitions into a continuous time Markov chain system
I have system which is mostly described by a CTMC (Continuous-time Markov chain) with a single absorbing state and a large but tractable and sparse transition matrix. However, at a fixed set of "...
- 21
2
votes
0
answers
111
views
Embedding a Markov chain in a Markov process
Let $X_{t\ge 0}$ be a Markov process with values in a metric space $(\mathcal{X},d)$ defined on a probabiltiy space $(\Omega,\mathcal{F},\mathbb{P})$ and let $(\tau_n)_{n=1}^{\infty}$ be a sequence of ...
- 21
0
votes
0
answers
34
views
Does the definition of mixing time work for general non-Markovian processes?
A definition of the mixing time for Markov chains is given by
\begin{equation}
\tau_{\text{mix}}\equiv\inf{\{t>0: \sup_i\left\vert \frac{\boldsymbol{p}(t|p_j(0)=\delta_{ij})}{\boldsymbol{\pi}}-\...
- 31
3
votes
0
answers
241
views
A few questions on Feller processes
Update. Most of my questions have been answered in the comments. I am adding these answers to the post.
There are at least three definitions of Feller semigroup and the corresponding processes: $C_0 \...
- 620
0
votes
0
answers
142
views
Calculating the expected hitting time of a specific birth and death chain
I am working with a specific birth and death chain, defined as follows.
Consider a set of states $X = \{0,1,2,...,n\}$, where $x^* \in (0,n)$ is a recurrent state. Transition probabilities are defined ...
0
votes
0
answers
161
views
Markov process with time varying transition kernels
I cross post this question from StackExchange as it may be more appropriate.
I am interested in studying the evolution of a variable $\alpha_t\in [0,1]$ governed by the following stochastic dynamical ...
3
votes
1
answer
307
views
"Ergodic theorem" for Markov kernels
Consider a discrete time Markov chain $(X_t)$ on a finite state space $\mathcal{S}$, with transition matrix $P$. Assume that the chain admits a stationary distribution $\pi$, which I will identify ...
3
votes
0
answers
54
views
Multi-type Galton-Watson-like process where only majority-type is allowed to reproduce
Are you aware of any research papers that have explored a multi-type Galton-Watson process in which only particles of the majority type are permitted to reproduce in each generation?
I've been unable ...
- 56
1
vote
1
answer
96
views
Asymptotic behavior of a Markov process on the set of $\{0,1\}$-polynomials
This question is cross-posted from https://math.stackexchange.com/questions/4711799/asymptotic-behavior-of-a-markov-process-on-the-set-of-0-1-polynomials
I am trying to study the asymptotic behavior ...
0
votes
0
answers
92
views
MDP Average Reward independent of Initial State
Consider a Markov Decision Process where the state space $S$ and the action space $A$ are continuous and compact.
In state $s$, if action $a$ is chosen and the next state becomes $s'$, the ...
- 115
2
votes
0
answers
155
views
Can a diffusion process admit an invariant measure with a non-differentiable density?
The precise domain of the generator $A$ of an Itō diffusion on a Hilbert space $H$ (assume $H=\mathbb R^d$, if that's easier for you to work with) can usually not be determined explicitly$^1$. Usually,...
- 167
1
vote
0
answers
115
views
Concatenation of Markov processes and independence
In chapter 14 of Sharpe's General Theory of Markov Processes the concatenation of Markov processes $X^1$ and $X^2$ is described. I've posed the relevant part at the bottom of this post.
It is rather ...
- 167
1
vote
0
answers
72
views
Bounding expectation of switching stochastic process
I am analyzing the behavior of an 1D stochastic dynamic system, where the state can vary randomly within a small magnitude. However, when the state deviates too much from zero, its expected magnitude ...
- 11
2
votes
1
answer
228
views
When is a stationary measure of a Markov chain "exponentially localized"?
Here exponentially localized can be thought in a non-rigorous manner as a measure that is mostly supported on a sparse number of nodes.
Some intuition can gained by thinking about a diffusion process, ...
- 2,281
3
votes
1
answer
340
views
Importance resampling with exponential weighting
Suppose that we have
$$
\frac{p(x)}{q(x)} \propto \exp(\tau f(x)),
$$
where we can sample from $q$ but not from $p$. Our goal is to generate a set of particles $\{x_i\}_{i=1}^n$ such that $n^{-1}\sum_{...
- 1,127
2
votes
1
answer
201
views
Joint irreducibility and aperiodicity of two independent Markov chains
Let $(X_i)_i, (Y_i)_i$ be two independent Markov chains on Polish state spaces $\mathcal{X}, \mathcal{Y}$ and with kernels $P, Q$. Suppose that they are both $\psi$-irreducible and aperiodic and have ...
- 203
4
votes
1
answer
186
views
Population growth with good and evil children - probability good outnumbers evil
Consider the following discrete-time population model. We start with a single "good" individual who reproduces asexually into $k$ children and dies in the process. At generation $t=2$, those ...
- 524
0
votes
1
answer
262
views
Construction of a Markov process with prescribed local behavior and state-dependent jump distribution
Let
$(E,\mathcal E)$ be a measurable space
$\mathcal E_b:=\left\{f:E\to\mathbb R\mid f\text{ is bounded and }\mathcal E\text{-measurable}\right\}$
$(\kappa_t)_{t\ge0}$ be a Markov semigroup on $(E,\...
- 167
0
votes
1
answer
96
views
What is the significance of Blumenthal and Getoor's result on the boundedness of paths of a standard Markov process?
In the book Markov processes and Potential Theory of Blumenthal and Getoor we can find the following result:
I don't understand the significance of this result. If I don't misinterpret the assertion, ...
- 167
3
votes
1
answer
474
views
Harmonic function and Markov chain
Let $X=(X_k)_{k \in \mathbb{N}}$ be a Markov chain with countable countable state space $S$ and transition matrix $P.$
Let $\mathcal{T}$ be the tail $\sigma$-field of $X:\mathcal{T}=\bigcap_{k \in \...
- 53
1
vote
1
answer
337
views
How can we determine the generator of this Markov process (at least formally)?
Let
$(\Omega,\mathcal A)$ be a measurable space;
$(E,\mathcal E)$ be a measurable space with $\{x\}\in\mathcal E$;
$(Y_t)_{t\ge0}$ be an $(E,\mathcal E)$-valued time-homogeneous Markov process on $(\...
- 167
0
votes
2
answers
804
views
Convergence of stationary distributions of a sequence of Markov Chains
I fairly new in the field of Stochastic Processes and Markov Chains so excuse my ignorance.
My question is: If we have a sequence of Markov chains such that each one has a stationary distribution $\pi^...
- 13
1
vote
1
answer
409
views
Occupation times for two-state Markov processes
Consider a two-state Markov process in continuous time, with states labelled $A$ and $B$. The transition rates for going from state $A$ to $B$, and state $B$ to $A$ are $\alpha$ and $\beta$ ...
0
votes
0
answers
72
views
If $\kappa$ is a Markov kernel with density $p$, does it generally hold $p(x,z)=\int p(x,y)p(y,z)\:{\rm d}y$?
Let $(E,\mathcal E)$ be a measurable space and $\kappa$ be a Markov kernel on $(E,\mathcal E)$. Assume that $$\kappa(x,B)=\int_Bp(x,y)\:\lambda({\rm d}y)\;\;\;\text{for all }(x,B)\in E\times\mathcal E$...
- 167
1
vote
1
answer
233
views
Random walks on Galton–Watson trees
I am working on a paper of Elie Aidekon : ‘Speed of the biased random walk on a Galton–Watson tree’ and have a question about one transformation in a proof:
\begin{align}
& 1+\frac{1}{1-\lambda}+\...
- 39
1
vote
0
answers
110
views
Birth and death process $M/M/\infty$
I was reading about continuous time Markov chains, when I met for the first time the theory of queue processes. In particular, I considered the following situation which I found on Wikipedia, called M/...
- 445
3
votes
2
answers
922
views
On representing a continuous time Markov chain by a stochastic integral of a Poisson random measure
Let $Q=(q_{ij})$ be the transition rate matrix of a continuous time Markov chain $\{ X_t \}$ with countable state space $M$. Let $q_i = -q_{ii}=\sum_{j \neq i}q_{ij}$, and let $\Gamma_{ij}$ be defined ...
- 31
1
vote
0
answers
181
views
Random walk on 2d lattice with obstacles
Consider a random work on $L=\mathbb Z^2$ endowed with obstacles (i.e each cell $(x,y)$ of $L$ may contain a obstacle, i.e the random walk halts whenever it hits such a cell). Let $P(x,y) = 1$ if cell ...
- 6,853
1
vote
2
answers
228
views
Is a linear combination of Markov generator a Markov generator?
Let $X$ be a compact metric set and $\mathcal{C}(X)$ be the set on continuous real functions over $X$ endowed with the supremum norm $\|\cdot\|_{\infty}$. Let $\Omega_1$ and $\Omega_2$ be two Markov ...
- 449
1
vote
1
answer
306
views
English translation of a Russian paper by Gordin and Lifšic
Unfortunately I can't read Russian, I was wondering if there is an English translation of this paper
“The central limit theorem for stationary Markov processes”, Dokl. Akad. Nauk SSSR, 239:4 (1978), ...
- 757
2
votes
2
answers
237
views
is this process a Markov one?
Here is the problem I can't solve.
Let $\xi_n$ $(n=1,2,3,\dots)$ be a sequence of i.i.d. random variables on $\mathbb{R}$ with density $p(x)>0$, let $\eta_n=\sum_{i=1}^{n}\xi_i^2$. Define $$\...
- 21
2
votes
1
answer
176
views
Monotonicity of Dirichlet form of Markov chain
Consider a continuous-time, irreducible Markov chain $X_t$ on a finite state space $E$. Assume the jump rates are $R(x,y)$ for $x,y\in E$, the generator is $L$, i.e for any function $f$ on E,
$$Lf(x)=\...
- 59
2
votes
1
answer
101
views
Preservation of the Markov Property under Conditioning
Let $(X_t,Z_t)_t$ be an $\mathbb{R}^{n}\times \mathbb{R}^m$-valued time-homogeneous Markov process on a filtered probability space $(\Omega,\mathcal{F},(\mathcal{F}_t)_t,\mathbb{P})$ with transition ...
- 77
2
votes
1
answer
205
views
Eigenspace of Gaussian Markov operator
Consider the (one-dimensional) Gaussian distribution $Q := N(\nu,\tau^2)$ and the (Gaussian) Markov operator
\begin{equation*}
\begin{array}{rccc}
R : & L_1(\mathbb{R},\mathcal{B}(\mathbb{R}),Q) &...
- 123
8
votes
4
answers
8k
views
Is there MDPs (Markov Decision Process) which have a non deterministic optimal policy?
I'm working on Markov Decision Process and I have not found yet an example of MDP that has a stochastic (non deterministic) optimal policy. Is there MDPs that have a stochastic optimal policy or is it ...
- 254
1
vote
0
answers
276
views
Path dependent Markov property
Let's consider a function $\Psi\in \mathcal{C}_B(\mathcal{C}[t,T])$ continuous and bounded
\begin{align*}
\Psi \colon \mathcal{C}[t,T] \longrightarrow [0,+\infty)
\end{align*}
Then my question is:...
- 159
0
votes
1
answer
203
views
Law of large numbers for Harris recurrent Markov chains
I'm trying to familiarize myself with the details of the proof that the Markov chains produce by the Metropolis-Hastings algorithm have a law of large numbers. I've found a half dozen or more ...
- 2,791
0
votes
0
answers
83
views
Constrained MDP
I have a question that is an extension of this one.
My question is: Can we say that for every policy, there exists a deterministic policy in case of a finite-state, finite-action infinite-horizon ...
- 154
2
votes
1
answer
114
views
Strong Data Processing Inequality for capped channels
Let $X$ and $Y$ be two $\rho$ correlated Gaussian vectors, such that $X,Y\sim N(0,1)^n$ and $E[X_iY_i]=\rho$.
Let $M_X = f(X)$ and $M_Y = f(Y)$ be $k$-bit functions of $X$ and $Y$, that is $H(X)=H(Y)=...
0
votes
0
answers
85
views
If $W$ is a Markov chain and $N$ is a Poisson process, then $\left(W_{N_t}\right)_{t\ge0}$ is Markov
Let $(\Omega,\mathcal A,\operatorname P)$ be a probability space, $(E,\mathcal E)$ be a measurable space, $(W_n)_{n\in\mathbb N_0}$ be a time-homogeneosu Markov chain on $(\Omega,\mathcal A,\...
- 167
1
vote
1
answer
175
views
Existence of Markov chain on nonnegative integers with specified rates
Let $\lambda_k,\mu_k\in\mathbb R_{\ge0}$ $(k\ge1)$ be nonnegative real numbers, let $S=\mathbb Z_{\ge0}$ be the nonnegative integers, let $T=\mathbb R_{\ge0}$ be the nonnegative real numbers and ...
- 276
1
vote
1
answer
173
views
Spectral gap of a Markov chain on the nonnegative integers
Let $\lambda_k,\mu_k\in\mathbb R_{\ge0}$ $(k\ge1)$ be nonnegative real numbers such that $\sum_{k=1}^\infty k\lambda_k<\infty,$ let $S=\mathbb Z_{\ge0}$ be the nonnegative integers, let $T=\mathbb ...
- 276
6
votes
2
answers
2k
views
Random walk to stay in an interval forever
Consider a random walk on the real time, starting from $0$. But this time assume that we can decide, for each step $i$, a step size $t_i>0$ to the left or the right with equal probabilities.
To ...
- 502