# Questions tagged [markov-chains]

The markov-chains tag has no usage guidance.

442
questions

**0**

votes

**0**answers

26 views

### Methods to find the spectrum of an operator

Suppose we have a bounded, self-adjoint operator $T$ on a set of functions $\mathcal{F}$. What kinds of methods are there to find the spectrum of $T$?
Here is the setting I'm wondering about: consider ...

**1**

vote

**0**answers

30 views

### The rate of convergence of Markov chain to stationary distribution

Let $X_t$ is Markov chain with transition rates $c: G \times G \rightarrow [0: +\infty)$, where $c(x, y) > 0$, $c(x, x) = -\sum_y c(x, y)$ for $x \neq y$. If $\mu_t(x)$ is the distribution of chain ...

**3**

votes

**0**answers

54 views

+100

### Rate of convergence of sojourn times of Markov chains

Let $(X_0,X_1,\dots)$ be a time-homogeneous Markov chain with finite state space $\Omega$.
Assume that $(X_0,X_1,\dots)$ is irreducible and aperiodic and let $\pi$ be its stationary distribution.
By ...

**1**

vote

**0**answers

59 views

### Stationary distribution of a Memoryless 2-type priority queue

I have come across the following priority queue, which seems quite natural to me.
A single queue with 2 types of costumers, independent Poisson arrivals and Poisson services.
First class costumers ...

**2**

votes

**1**answer

116 views

### The reference on Markov chains uncovering the power of the subject in a better way for a working macro-economist

This is by no means a research question. But asking here I hope for the most expert opinion.
A friend of mine, who is a working economist, asked me for advice about a book which uncovers wealth and ...

**6**

votes

**1**answer

142 views

### Perron-Frobenius and Markov chains on countable state space

The following question naturally arises in the theory of Markov chains with countable state space to which I would be curious to know the answer:
Let $A:\ell^1 \rightarrow \ell^1$ be a contraction, i....

**0**

votes

**0**answers

57 views

### convergence rate for ergodic Markov chains induced by stable dynamical systems

Consider a deterministic dynamical system on $\mathbb{R}^n$ defined by the recurrence $x_{t+1} = f(x_t)$.
Suppose the dynamical system is stable in the following sense: there exists a $Q : \mathbb{R}^...

**1**

vote

**2**answers

122 views

### Is a linear combination of Markov generator a Markov generator?

Let $X$ be a compact metric set and $\mathcal{C}(X)$ be the set on continuous real functions over $X$ endowed with the supremum norm $\|\cdot\|_{\infty}$. Let $\Omega_1$ and $\Omega_2$ be two Markov ...

**0**

votes

**0**answers

36 views

### Distribution of total offspring of Poisson multitype branching process

The question I have is related to the question asked here: Total offspring of Poisson multitype branching process
Fix $d\in\mathbb{N}$ and let $Z_n\in\mathbb{N}^d$ be a multitype branching process, ...

**-2**

votes

**1**answer

42 views

### Stationary distribution of a weighted directed acyclic graph

Is there any way to calculate the equilibrium (stationary) distribution for a weighted directed acyclic graph?
Some references emphasized adjacency matrix to be symmetric.
https://arxiv.org/abs/1012....

**0**

votes

**0**answers

128 views

### Continuous-time random walk on $\mathbb{R}$ that never stays still

Consider a walker on the real line $\mathbb{R}$ and two probability density functions $w$ and $j$ defined over $\mathbb{R}$.
A walker starts at $0$ and iterates the following: it samples a waiting ...

**1**

vote

**0**answers

36 views

### Stationary distributions of convex combination of stochastic matrices

Consider two irreducible finite state Markov chains with transition matrices $A,B\in\mathbb{R}^{n\times n}$.
Let $x$ and $y$ be the unique stationary distributions of $A$ and $B$, respectively.
Now ...

**0**

votes

**0**answers

28 views

### Markov chains with “clustered” stationary distributions

Are there any canonical or well-known Markov chains whose stationary distributions are basically clustered into two or more components? Obviously, it is easy to create one, but I’m wondering if there ...

**0**

votes

**0**answers

18 views

### Estimation of probability matrix from samples at different time intervals

I am given discrete-time Markov chain that evolves on a finite subset $\{1,\dots,n\}$. This Markov chain is time-homogeneous and has a transition matrix $P$ that I want to estimate.
Let $X_t$ be the ...

**2**

votes

**1**answer

116 views

### Monotonicity of Dirichlet form of Markov chain

Consider a continuous-time, irreducible Markov chain $X_t$ on a finite state space $E$. Assume the jump rates are $R(x,y)$ for $x,y\in E$, the generator is $L$, i.e for any function $f$ on E,
$$Lf(x)=\...

**0**

votes

**0**answers

23 views

### optimal policies in Markov Decision Process

I recently did a course in Markov Decision Process from Puterman. In the 6th chapter, it is stated that in a finite state, finite action, infinite-horizon discounted MDP, where the costs and ...

**0**

votes

**0**answers

56 views

### Constrained MDP

I have a question that is an extension of this one.
My question is: Can we say that for every policy, there exists a deterministic policy in case of a finite-state, finite-action infinite-horizon ...

**1**

vote

**0**answers

96 views

### Harnack inequalities for Markov chains

We consider a (continuous time) Markov chain $X=(\{X_t\}_{t \ge 0},\{P_x\}_{x \in V})$ on a finite set $V$. We assume moreover that $V$ is embedded into $\mathbb{R}^d$. The generator $\mathcal{L}$ of $...

**0**

votes

**1**answer

99 views

### Sets of invariant measures of Markov operators

A family of Markov operators $P_i \colon C \to C, i \in I$ is given. Let $V_i$ be the set of the $P_i$-invariant measures. Is there any result in the literature about a necessary and sufficient ...

**0**

votes

**0**answers

42 views

### Exit time distributions for jump type Markov processes

This question is about an estimate for some jump-type Markov processes.
Let $X=(\{X_t\}_{t \ge0},\{P_x\}_{x \in \mathbb{R}^d})$ be a symmetric $\alpha$-stable process on $\mathbb{R}^d$. We know that ...

**0**

votes

**0**answers

68 views

### If $W$ is a Markov chain and $N$ is a Poisson process, then $\left(W_{N_t}\right)_{t\ge0}$ is Markov

Let $(\Omega,\mathcal A,\operatorname P)$ be a probability space, $(E,\mathcal E)$ be a measurable space, $(W_n)_{n\in\mathbb N_0}$ be a time-homogeneosu Markov chain on $(\Omega,\mathcal A,\...

**1**

vote

**1**answer

77 views

### If $L_t=\sum_{i=1}^{N_t}Y_i$ is a compound Poisson process, then $\left|\left\{s\in[0,t]:\Delta L_s\in B\right\}\right|=\sum_{i=1}^{N_t}1_B(Y_i)$

Let $H$ be a $\mathbb R$-Hilbert space, $\mu$ be a finite measure on $\mathcal B(H)$ with $\mu(\{0\})=0$ and $(L_t)_{t\ge0}$ be a $H$-valued càdlàg Lévy process on a probability space $(\Omega,\...

**0**

votes

**0**answers

92 views

### Path dependent Markov property

Let's consider a function $\Psi\in \mathcal{C}_B(\mathcal{C}[t,T])$ continuous and bounded
\begin{align*}
\Psi \colon \mathcal{C}[t,T] \longrightarrow [0,+\infty)
\end{align*}
Then my question is:...

**1**

vote

**1**answer

149 views

### finiteness of moments of the stationary distribution of a Markov chain

I have a Markov chain $\{X_k\}_{k\geq 0}$ on $\mathbb{R}$. The corresponding probability density functions satisfy
$$
f_{k+1}(t) = \int_{-\infty}^\infty \Psi(t,\tau)f_k(\tau)\,d\tau,\qquad k=0,1,2,\...

**1**

vote

**0**answers

36 views

### Chernoff-type Bounds for Continuous-space Markov Chains

Let $X_1, X_2, \dots, X_n$ be $n$ samples from a discrete-time continuous-space Markov Chain.
Are there any good references who have provided a Chernoff-type bound regarding the behaviour of the ...

**1**

vote

**1**answer

60 views

### Defining measures through products of Markov kernels

I am quite puzzled by the expression given in equation 21 (page 10) in this paper,
https://arxiv.org/pdf/1802.09188.pdf
Its LHS seems to be a measure $\nu_n^N$ and hence I guess it takes as argument ...

**0**

votes

**0**answers

28 views

### Spectral gap of continuous-time Markov chain on nonnegative integers: The geometric long indel length chain

Let $S=\mathbb Z_{\ge0}$ be the nonnegative integers, and let $T=\mathbb R_{\ge0}$ be the nonnegative real numbers. Next, let $\gamma\in(0,1),r\in(0,1),$ and let $Q=(Q_{n,m})_{n,m\in S}$ be such that ...

**0**

votes

**0**answers

35 views

### Distribution of hitting time of set of states with all $1$s for continuous-time Markov chain on binary strings of length $\le\! n$

Let $n\in\mathbb Z_{\ge1}$ be a strictly positive integer, let $T=\mathbb R_{\ge0}$ be the nonnegative real numbers, let $S=\cup_{m=0}^n\{0,1\}^m,$ let $\mu_1,\dots,\mu_n\in\mathbb R_{\ge0}$ be ...

**3**

votes

**0**answers

50 views

### Mixing times for the exclusion process with rejection

Consider the following Markov chain on $k$-subsets of $\{1,\ldots, L\}$, equivalently, sequences $x\in \{0,1\}^L$ with $k$ 1's.
Let $p_1,\ldots, p_L\in (0,1)$ and $q_i=1-p_i$.
At each step, choose an ...

**2**

votes

**1**answer

73 views

### Strong Data Processing Inequality for capped channels

Let $X$ and $Y$ be two $\rho$ correlated Gaussian vectors, such that $X,Y\sim N(0,1)^n$ and $E[X_iY_i]=\rho$.
Let $M_X = f(X)$ and $M_Y = f(Y)$ be $k$-bit functions of $X$ and $Y$, that is $H(X)=H(Y)=...

**1**

vote

**0**answers

39 views

### SDP relaxation vs. Monte Carlo for MaxCut: which one performs better?

the Goemans Williamson SDP relaxation of the MAXCUT problem famously gives a polynomial approximation ratio of .87856 for the MAXCUT on regular graphs.
Another popular approach to obtain efficient ...

**1**

vote

**0**answers

41 views

### Convergence of empirical measure to Mc-Kean Vlasov equation for mean-field model with jumps

I am interested in the following mean-field model introduced in the reference below:
There are $N$ particles. At each instant of time, a particle's state is a particular value taken from the finite ...

**2**

votes

**0**answers

43 views

### Representing a continuous time-inhomogeneous Markov chain by a stochastic integral

I am interested in the following mean-field model introduced in the reference below:
There are $N$ particles. At each instant of time, a particle's state is a particular value taken from the finite ...

**1**

vote

**1**answer

146 views

### If a Markov semigroup is eventually contractive, can we conclude that it admits a unique invariant measure?

Let $E$ be a separable $\mathbb R$-Banach space, $\rho$ be a complete separable metric on $E$, $\operatorname W_\rho$ denote the Wasserstein metric of order $1$ associated to $\rho$, $\mathcal M_1(E)$ ...

**1**

vote

**1**answer

114 views

### Spectral gap of a Markov chain on the nonnegative integers

Let $\lambda_k,\mu_k\in\mathbb R_{\ge0}$ $(k\ge1)$ be nonnegative real numbers such that $\sum_{k=1}^\infty k\lambda_k<\infty,$ let $S=\mathbb Z_{\ge0}$ be the nonnegative integers, let $T=\mathbb ...

**2**

votes

**1**answer

143 views

### English translation of a Russian paper by Gordin and Lifšic

Unfortunately I can't read Russian, I was wondering if there is an English translation of this paper
“The central limit theorem for stationary Markov processes”, Dokl. Akad. Nauk SSSR, 239:4 (1978), ...

**1**

vote

**2**answers

127 views

### Of all probability matrix $P$ having stationary distribution $\pi$, find the one having smallest diagonal

Hello MathOverflow community,
I am requesting your help today trying to solve a somewhat odd problem. Is there a way to find through some numerical algorithm such as Newton's method the stochastic ...

**1**

vote

**2**answers

159 views

### Extension of spectral gap inequality in Wasserstein distance

Let $E$ be a separable $\mathbb R$-Banach space, $\rho_r$ be a metric on $E$ for $r\in(0,1]$ with $\rho_r\le\rho_s$ for all $0<r\le s\le1$, $\rho:=\rho_1$, $$d_{r,\:\delta,\:\beta}:=1\wedge\frac{\...

**2**

votes

**0**answers

25 views

### If a stochastic flow is Fréchet differentiable in the spatial parameter, does the induced transition semigroup preserve differentiability?

Let $(\Omega,\mathcal A,\operatorname P)$ be a probability space, $(E,\mathcal E)$ be a measurable space, $X:\Omega\times[0,\infty)\times E\to E$ be $(\mathcal A\otimes\mathcal B([0,\infty))\otimes\...

**3**

votes

**0**answers

61 views

### How does one define the gradient of a Markov semigroup?

In the context of functional inequalities for Markov semigroups $(\mathcal P_t)_{t\ge0}$, what is one denoting by $\nabla\mathcal P_tf$? For example, I've found the following assumption in this paper:
...

**1**

vote

**1**answer

92 views

### Existence of Markov chain on nonnegative integers with specified rates

Let $\lambda_k,\mu_k\in\mathbb R_{\ge0}$ $(k\ge1)$ be nonnegative real numbers, let $S=\mathbb Z_{\ge0}$ be the nonnegative integers, let $T=\mathbb R_{\ge0}$ be the nonnegative real numbers and ...

**2**

votes

**1**answer

267 views

### Calculate Radon-Nikodym derivative

For the laws of two pure-jump Markov processes $\mu_1$ and $\mu_2$ on $\mathbb R^n$, which generators are
$H_1f(x)=\int h(x,dy) (f(y)-f(x))$
and $H_2f(x)=\int e^{-g(x,y)} h(x,dy) (f(y)-f(x))$ (...

**2**

votes

**1**answer

119 views

### Eigenspace of Gaussian Markov operator

Consider the (one-dimensional) Gaussian distribution $Q := N(\nu,\tau^2)$ and the (Gaussian) Markov operator
\begin{equation*}
\begin{array}{rccc}
R : & L_1(\mathbb{R},\mathcal{B}(\mathbb{R}),Q) &...

**2**

votes

**2**answers

172 views

### is this process a Markov one?

Here is the problem I can't solve.
Let $\xi_n$ $(n=1,2,3,\dots)$ be a sequence of i.i.d. random variables on $\mathbb{R}$ with density $p(x)>0$, let $\eta_n=\sum_{i=1}^{n}\xi_i^2$. Define $$\...

**1**

vote

**1**answer

72 views

### Conditions for optimal stationary strategies in MDPs

I have a specific Markov decision process (MDP) which is generated from a problem in another domain. What I would like to show is that under the limit of means criterion (no discounting) the optimal ...

**0**

votes

**0**answers

18 views

### Calculate regime-switching correlation matrix without assumption on distribution

There's rich literature on Markov regime-switching dynamic correlation matrix. But most results seem to assume a certain kind of distribution and use MLE/EM. For example, some sort of multivariate ...

**6**

votes

**2**answers

538 views

### what is the number of paths returning to 0 on the hexagonal lattice

I am looking for an estimation of the number of paths of length $n$ going from 0 to 0 on the hexagonal (or honeycomb) lattice.
I can find plenty on references on self avoiding paths, but I am looking ...

**3**

votes

**0**answers

52 views

### Algebraic property of a transition matrix

Consider the simple random walk on $\mathbb{Z}^2$. Given a finite $\Sigma \subset \mathbb{Z}^2$, one can induce the random walk on $\Sigma$: set $\tau_0 = 0$, and define recursively $\tau_{n+1} := \...

**1**

vote

**0**answers

109 views

### Upper and lower bounds on the entries of a matrix power

Say I have a non-negative square $n\times n$ irreducible stochastic matrix $A$ (i.e., each column sums to 1), for which the following holds:
$$A_{ij} > 0 \iff A_{ji} > 0.$$
I know that no more ...

**0**

votes

**1**answer

45 views

### Friedrich's extension of the generator of a continuous time markov chaoin

Consider the infinitesimal generator $G$ of a Markov chain with state space $\mathbb{Z}$ such that it is symmetric with respect to a measure $\mu$ on $\mathbb{Z}$. Then, the operator $(G,C_c(\mathbb{Z}...