All Questions
Tagged with pr.probability markov-chains
23 questions
5
votes
2
answers
516
views
Anticoncentration of the convolution of two characteristic functions
Edit: This is a question related to my other post, stated in a much more concrete way I think.
I am interested in anything (ideas, references) related to the following problem:
Suppose that $A \...
- 696
1
vote
1
answer
143
views
Comparison of hitting probability of two Markov chains both with only one absorbing state version 2 under stronger condition
Let $N_n:=\{1,2,\cdots,n\}$. Given two finite states Markov chains $\big(X^{(j)}_i\in N_n\}\big)_{i=0}^\infty$ for $j\in\{1,2\}$, both of which have two absorbing states at $1$ and $n$.
$\text{Pr}\...
- 2,239
19
votes
5
answers
18k
views
Time-inhomogeneous Markov chains
I'm trying to find out what is known about time-inhomogeneous ergodic Markov Chains where the transition matrix can vary over time. All textbooks and lecture notes I could find initially introduce ...
- 191
12
votes
3
answers
1k
views
How to sample a uniform random polyomino?
A polyomino is formed by joining finitely many unit squares edge to edge. It may be regarded as a finite subset of the regular square tiling with a connected interior. In particular, for us, ...
- 7,857
8
votes
4
answers
8k
views
Is there MDPs (Markov Decision Process) which have a non deterministic optimal policy?
I'm working on Markov Decision Process and I have not found yet an example of MDP that has a stochastic (non deterministic) optimal policy. Is there MDPs that have a stochastic optimal policy or is it ...
- 254
7
votes
1
answer
337
views
First Collision Time for k Random Walkers on a Torus
I consider $k$ random walkers on $\mathbb{Z}^{d}/n \mathbb{Z}^{d}$, the $d$-dimensional torus of side length $n$. More precisely, I will define a Markov chain $Z_{t} = (X_{t}[1], \ldots, X_{t}[k])$ ...
- 71
7
votes
1
answer
391
views
Idempotent splitting for Markov kernels
Let $X$ be a standard Borel space and $e : X \to X$ a Markov kernel. Suppose that $e$ is idempotent, that is $e \circ e = e$, or written out using the Chapman-Kolmogorov equation,
$$e(A|x) = \int_X e(...
- 6,406
4
votes
0
answers
589
views
Optimal transport between two distributions in a Markov chain
In a previous question, given an ergodic Markov chain, I'm interesting in sampling as short a path as possible with prescribed distributions for its endpoints. In a comment, I propose that the ...
- 6,853
3
votes
2
answers
973
views
How much larger than the relaxation time can the mixing time be?
The notation is mostly taken from the book "Markov chains and mixing times" by Levin, Peres, and Wilmer.
Consider an irreducible, aperiodic, time-reversible, discrete-time Markov chain on a finite ...
- 1,269
3
votes
1
answer
281
views
Drunkards Uphill Walk
An urn is filled with n black and n white balls. Do the following Markov process: 1. Draw ball, memorize color, throw it back. 2. Draw another ball (might be the same!), color it with memorized color, ...
- 4,829
3
votes
0
answers
494
views
Maximization of a total variation distance subject to another total variation distance in Markov chain
Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the ...
- 1,109
3
votes
2
answers
264
views
Probability of one species reaching zero before the other in a Markov process on a 2d lattice
$\textbf{Background}$: Say we've got a two-variable system of stochastic chemical reactions, with quantities $\vec{x}(t) = (x_1(t),x_2(t)) \in \mathbb{N}^2$ evolving according to the following system, ...
- 41
2
votes
1
answer
640
views
Reachability for Markov process
Let $X$ be a Markov process (in continuous or discrete time) and define an event
$$
R(T,A) = (\exists t\leq T: X_t \in A).
$$
I have seen in one paper that
$$
\Pr[R(\infty,A)] = \sup\limits_{\tau} \...
- 1,655
1
vote
1
answer
121
views
Characterization of Fellerian kernels
This question concerns Feller Markov kernels, similar to Vanessa's question.
Terminology
By 'Markov kernel' $N:E\to F$, we adopt exactly the same definition as Vanessa, with the exception that $E,F$ ...
- 235
1
vote
1
answer
352
views
Double Markovity
Suppose we have a double Markov relation for three random variables $X$, $Y$ and $W$ as follows
$$X\to W\to Y,$$ and $$X\to Y\to W.$$
How to prove that there exist functions $f$ and $g$ such that
$$...
- 1,109
1
vote
1
answer
127
views
Almost certain extinction for a Markov Jump Process
I'm studying a simplification of a biological neuron model with $n$ neurons. We are describing the evolution of the membrane potential of each neuron. Let $(X_t)_{t\geq 0}$ be a Markov Jump Process in ...
- 203
1
vote
1
answer
561
views
What do you call a Markov kernel continuous w.r.t. the weak topology?
Let $X$ and $Y$ be Polish spaces and $K$ a Markov kernel from $X$ to $Y$. That is, $K$ is a mapping $X \times \mathcal{B}_Y \rightarrow [0,1]$ (where $\mathcal{B}_Y$ is the $\sigma$-algebra of Borel ...
- 1,368
1
vote
1
answer
119
views
Comparison of hitting probability of two Markov chains both with only one absorbing state
Let $N_n:=\{1,2,\cdots,n\}$. Given two finite states Markov chains $\big(X^{(j)}_i\in N_n\}\big)_{i=0}^\infty$ for $j\in\{1,2\}$, both of which have one absorbing state $1$.
Pr$(X^{(1)}_{i+1}=1|X_i=1)...
- 2,239
0
votes
1
answer
262
views
Construction of a Markov process with prescribed local behavior and state-dependent jump distribution
Let
$(E,\mathcal E)$ be a measurable space
$\mathcal E_b:=\left\{f:E\to\mathbb R\mid f\text{ is bounded and }\mathcal E\text{-measurable}\right\}$
$(\kappa_t)_{t\ge0}$ be a Markov semigroup on $(E,\...
- 167
0
votes
1
answer
95
views
If a probability measure is stationary in both forward time and reverse time, does this imply that the measure is incompressible?
Let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space and let $X$ be a compact metric space. Let $F \colon \Omega \times X \to X$ and $\bar{F} \colon \Omega \times X \to X$ be measurable ...
- 708
0
votes
1
answer
82
views
In smooth stochastic dynamics, if a Lebesgue-like measure is both forward-time and reverse-time stationary, is the measure necessarily incompressible?
Let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space and let $X$ be a compact connected $C^\infty$-smooth manifold. Let $F \colon \Omega \times X \to X$ and $\bar{F} \colon \Omega \times X \to ...
- 708
0
votes
1
answer
96
views
What is the significance of Blumenthal and Getoor's result on the boundedness of paths of a standard Markov process?
In the book Markov processes and Potential Theory of Blumenthal and Getoor we can find the following result:
I don't understand the significance of this result. If I don't misinterpret the assertion, ...
- 167
-1
votes
1
answer
370
views
What's the probability of two independent events in time domain?
Suppose there are two independent events A and B. The probability that A or ...
- 29