All Questions
35 questions
1
vote
0
answers
114
views
An urn model with weighted objects and replacement
Consider the following game:
In an urn, there are $K$ balls, $x_0$ of them are blue and light (mass $m_0$), $x_1$ are blue and heavy ($m_1$), $x_2$ are red and light ($m_2$), the rest $x_3$ are red ...
- 31
0
votes
0
answers
92
views
MDP Average Reward independent of Initial State
Consider a Markov Decision Process where the state space $S$ and the action space $A$ are continuous and compact.
In state $s$, if action $a$ is chosen and the next state becomes $s'$, the ...
- 115
1
vote
1
answer
99
views
Asymptotic variance for averages of trajectory functionals of Markov chain
I am looking for references on theory for convergence rates of ergodic averages of a Markov chain in the more general setting where the functional is over multiple states or even a whole trajectory, ...
- 51
0
votes
0
answers
161
views
Markov process with time varying transition kernels
I cross post this question from StackExchange as it may be more appropriate.
I am interested in studying the evolution of a variable $\alpha_t\in [0,1]$ governed by the following stochastic dynamical ...
1
vote
1
answer
96
views
Asymptotic behavior of a Markov process on the set of $\{0,1\}$-polynomials
This question is cross-posted from https://math.stackexchange.com/questions/4711799/asymptotic-behavior-of-a-markov-process-on-the-set-of-0-1-polynomials
I am trying to study the asymptotic behavior ...
7
votes
1
answer
391
views
Idempotent splitting for Markov kernels
Let $X$ be a standard Borel space and $e : X \to X$ a Markov kernel. Suppose that $e$ is idempotent, that is $e \circ e = e$, or written out using the Chapman-Kolmogorov equation,
$$e(A|x) = \int_X e(...
- 6,406
3
votes
1
answer
220
views
Carne-Varopoulos bound and stationary measure
Let $\Gamma$ denote the Cayley graph for a finitely generated group $G$, and let $p_n(x, y)$ denote the transition probability that a random walk starting at $x$ reaches $y$ at time $n$. A famous &...
- 31
8
votes
1
answer
691
views
Probabilistic proof for derivative of invariant distribution of a Markov chain
Let $P$ be an irreducible Markov matrix, and $\pi$ its stationary distribution. Let $D$ be a perturbation matrix which is zero except for two entries in row $r$:
$$D_{rg}=+1 \qquad D_{r\ell}=-1.$$
Let ...
- 1,068
0
votes
2
answers
804
views
Convergence of stationary distributions of a sequence of Markov Chains
I fairly new in the field of Stochastic Processes and Markov Chains so excuse my ignorance.
My question is: If we have a sequence of Markov chains such that each one has a stationary distribution $\pi^...
- 13
1
vote
0
answers
332
views
Markov chains with drift
We consider a Markov process $X$ on a finite set $\mathcal{X} (\neq \emptyset)$. Basically, $X$ is associated with a generator of the following form
\begin{align*}
Af(x)=\lambda(x)\sum_{ y\in \mathcal{...
- 721
2
votes
1
answer
134
views
The reference on Markov chains uncovering the power of the subject in a better way for a working macro-economist
This is by no means a research question. But asking here I hope for the most expert opinion.
A friend of mine, who is a working economist, asked me for advice about a book which uncovers wealth and ...
- 774
1
vote
1
answer
306
views
English translation of a Russian paper by Gordin and Lifšic
Unfortunately I can't read Russian, I was wondering if there is an English translation of this paper
“The central limit theorem for stationary Markov processes”, Dokl. Akad. Nauk SSSR, 239:4 (1978), ...
- 757
0
votes
2
answers
128
views
Markov with epsilon memory and Quantitative Strong Markov property
We have a process $\{X_{t}\}_{t\geq 0}$ ,with fixed parameter $\epsilon>0$, starting from zero that satisfies
The process is strictly monotone $X_{t+r}-X_{t}>0$ with moments existing $p\in(-\...
- 5,474
2
votes
0
answers
113
views
Characterizing the relationship between element-wise Markov transitions and the full-conditionals of the stationary distribution
Consider a $p$ dimensional random variable with a discrete support. Consider a Markov transition kernel on the state space that is defined in terms of element-wise transition distributions.
One can ...
- 2,791
5
votes
1
answer
199
views
Reference request: When is the variance in the central limit theorem for Markov chains positive?
I'm looking for a reference which gives sufficient conditions for the variance to be positive in the central limit theorem for Markov chains (cf https://en.wikipedia.org/wiki/...
- 153
3
votes
0
answers
88
views
Joint drunkard walks
The drunkard walk is a game where two players have $a$ and $b$ dollars, respectively, and they play a series of fair games (both risking one dollar in each game) until one of them goes broke.
My ...
- 131
4
votes
2
answers
206
views
Reference on a markov chain / Queue
Im looking for a reference that treats the Markov Chain defined by
$$W_i=(W_{i-1}-1)\vee X_i$$
where $X_i\geq 0$ are i.i.d discrete variables. In particular im interested in a reference that treats ...
- 315
1
vote
0
answers
120
views
Existence of Time-Reversed Markov Kernels
Suppose I have a probability measure $\pi$ and a Markov kernel $q$ which leaves $\pi$ invariant, in the sense that
\begin{align}
\int_x \pi(dx) q(x \to dy) = \pi(dy)
\end{align}
Then, a (the) time-...
- 801
2
votes
0
answers
37
views
Reference request: semimarkov processes
What are some good modern introductions to the theory of semimarkov processes? To be clear, by a semimarkov processes, I mean a Markov chain, together with "waiting times" between transitions, the ...
- 505
0
votes
0
answers
72
views
Invariant measures for a renewal process driven by Interarrival times bounded away from zero
Good morning, I apologize in advance if my question sounds too basic but after some research I was unable to come up with satisfactory answers to my doubts.
I am currently studying a model which ...
- 277
3
votes
2
answers
436
views
Central limit theorem for weak dependent bernoulli random variables
Suppose $\epsilon_1,\epsilon_2,...$ are i.i.d bounded random variables with compact support. Let $X_k=g_k(\epsilon_k,...,\epsilon_1)$ be Bernoulli random variables with the covariance between $X_i$ ...
- 339
1
vote
1
answer
65
views
Transition of probability in duality and its properties
Let $(E,\mathscr{E})$ be a measurable space. Two transition of probabilities
$p, q\colon E\times\mathscr{E}\to [0,1]$ are said to be in duality relative to
a probability measure $m$ if for every ...
- 95
1
vote
1
answer
170
views
Comprehensive reference for lumped or projected markov chains
Consider a Markov chain $X_n$ taking values in finite countable set $\mathcal{X}$ with transition matrix $P$. Consider a function $f:\mathcal{X}\to\mathcal{Y}$ inducing a partition $\mathcal{Y}=\{\...
- 133
3
votes
0
answers
106
views
Find the generator of a markov process with constant decay and exponential jumps
Suppose we have a continuous time Markov process $(X_t)_{t\in [0,\infty)}$. This Markov process represents the queue length in amount of work left, therefore its state space is given as $S = [0,\infty)...
- 277
4
votes
1
answer
176
views
Random Walk with "Forward Dependency"
Let $\{X_t\}_{t=-\infty}^{\infty}$ be a sequence of random variables. We are interested in a "random walk" (or more generally, a random field) that can be characterized by
$$
X_t ~|~ X_{t-k}, \ldots, ...
- 1,127
2
votes
0
answers
159
views
Distribution of path probabilities for a finite absorbing Markov chain
I am interested in the distribution of path probabilities for a finite
absorbing (but otherwise well behaved) Markov chain. Has this topic
been considered in the literature?
A bit of Googling ...
- 15.4k
1
vote
1
answer
404
views
Does Irreducibility holds for the Ergodic non-stationary Markov chain?
In the stationary case, I know that if the chain is irreducible and aperiodic, it is Ergodic. But in the non-stationary case, i can not comprehend the content deeply. I want to know if Irreducibility ...
- 121
3
votes
1
answer
143
views
Markov chain dichotomy
Suppose $X$ is a complete separable metric space, and there is a continuous map $x \mapsto \mu_x$ associating to each point in $X$ a probability measure on $X$ (where we use the weak topology on the ...
- 4,304
1
vote
1
answer
140
views
Reference request: Cover times, Mixing Times and DGFF applied in statistics?
I am trying to find if in active research in statistics, there is interest in mixing times, cover times of graphs, and/or the discrete Gaussian free field?
I haven't found anything so far for the ...
- 11
2
votes
0
answers
74
views
Literature/Book on counting processes
I seek literature that makes a rigorous treatment of counting processes. In particular im interested in a precise treatment of the conditional intensity $\lambda_t$ which is often informally defined ...
- 315
3
votes
0
answers
144
views
The spring Markov chain on $\mathbb{N}$
I'm trying to understand and learn more about "almost surely bounded" Markov chains on countable state spaces. I'm looking for references where I can learn how to work with more complicated examples ...
- 1,961
2
votes
1
answer
2k
views
Stationary distribution for time-inhomogeneous Markov process
I have a two state, discrete time, time-inhomogeneous Markov process with transition matrix defined by
$$T_i=\begin{pmatrix}
1-p_i\alpha & p_i\alpha \\
p_i\beta& 1-p_i\beta
\end{pmatrix}$$
...
- 23
19
votes
5
answers
18k
views
Time-inhomogeneous Markov chains
I'm trying to find out what is known about time-inhomogeneous ergodic Markov Chains where the transition matrix can vary over time. All textbooks and lecture notes I could find initially introduce ...
- 191
0
votes
1
answer
2k
views
Stationary distribution in general Markov Chains
This is just a reference request for a result which is very general, useful and should be well-known, but I've failed to find a good reference to cite.
The problem is to define the "most natural" ...
- 1,341
6
votes
1
answer
218
views
Finding cohesive (low exit probability) sets in a Markov process
The following is a fact about Markov chains that came up in a game theory paper. The purpose of this question is to ask if related notions or similar results are found elsewhere in probability, or are ...
- 1,068