# Questions tagged [markov-chains]

The tag has no usage guidance.

442 questions
Filter by
Sorted by
Tagged with
26 views

### Methods to find the spectrum of an operator

Suppose we have a bounded, self-adjoint operator $T$ on a set of functions $\mathcal{F}$. What kinds of methods are there to find the spectrum of $T$? Here is the setting I'm wondering about: consider ...
30 views

### The rate of convergence of Markov chain to stationary distribution

Let $X_t$ is Markov chain with transition rates $c: G \times G \rightarrow [0: +\infty)$, where $c(x, y) > 0$, $c(x, x) = -\sum_y c(x, y)$ for $x \neq y$. If $\mu_t(x)$ is the distribution of chain ...
54 views
+100

### Rate of convergence of sojourn times of Markov chains

Let $(X_0,X_1,\dots)$ be a time-homogeneous Markov chain with finite state space $\Omega$. Assume that $(X_0,X_1,\dots)$ is irreducible and aperiodic and let $\pi$ be its stationary distribution. By ...
59 views

### Stationary distribution of a Memoryless 2-type priority queue

I have come across the following priority queue, which seems quite natural to me. A single queue with 2 types of costumers, independent Poisson arrivals and Poisson services. First class costumers ...
116 views

### The reference on Markov chains uncovering the power of the subject in a better way for a working macro-economist

This is by no means a research question. But asking here I hope for the most expert opinion. A friend of mine, who is a working economist, asked me for advice about a book which uncovers wealth and ...
142 views

### Perron-Frobenius and Markov chains on countable state space

The following question naturally arises in the theory of Markov chains with countable state space to which I would be curious to know the answer: Let $A:\ell^1 \rightarrow \ell^1$ be a contraction, i....
57 views

39 views

### SDP relaxation vs. Monte Carlo for MaxCut: which one performs better?

the Goemans Williamson SDP relaxation of the MAXCUT problem famously gives a polynomial approximation ratio of .87856 for the MAXCUT on regular graphs. Another popular approach to obtain efficient ...
41 views

### Convergence of empirical measure to Mc-Kean Vlasov equation for mean-field model with jumps

I am interested in the following mean-field model introduced in the reference below: There are $N$ particles. At each instant of time, a particle's state is a particular value taken from the finite ...
43 views

### Representing a continuous time-inhomogeneous Markov chain by a stochastic integral

I am interested in the following mean-field model introduced in the reference below: There are $N$ particles. At each instant of time, a particle's state is a particular value taken from the finite ...
146 views

### If a Markov semigroup is eventually contractive, can we conclude that it admits a unique invariant measure?

Let $E$ be a separable $\mathbb R$-Banach space, $\rho$ be a complete separable metric on $E$, $\operatorname W_\rho$ denote the Wasserstein metric of order $1$ associated to $\rho$, $\mathcal M_1(E)$ ...
114 views

Let $\lambda_k,\mu_k\in\mathbb R_{\ge0}$ $(k\ge1)$ be nonnegative real numbers such that $\sum_{k=1}^\infty k\lambda_k<\infty,$ let $S=\mathbb Z_{\ge0}$ be the nonnegative integers, let $T=\mathbb ... 1answer 143 views ### English translation of a Russian paper by Gordin and Lifšic Unfortunately I can't read Russian, I was wondering if there is an English translation of this paper “The central limit theorem for stationary Markov processes”, Dokl. Akad. Nauk SSSR, 239:4 (1978), ... 2answers 127 views ### Of all probability matrix$P$having stationary distribution$\pi$, find the one having smallest diagonal Hello MathOverflow community, I am requesting your help today trying to solve a somewhat odd problem. Is there a way to find through some numerical algorithm such as Newton's method the stochastic ... 2answers 159 views ### Extension of spectral gap inequality in Wasserstein distance Let$E$be a separable$\mathbb R$-Banach space,$\rho_r$be a metric on$E$for$r\in(0,1]$with$\rho_r\le\rho_s$for all$0<r\le s\le1$,$\rho:=\rho_1$, $$d_{r,\:\delta,\:\beta}:=1\wedge\frac{\... 0answers 25 views ### If a stochastic flow is Fréchet differentiable in the spatial parameter, does the induced transition semigroup preserve differentiability? Let (\Omega,\mathcal A,\operatorname P) be a probability space, (E,\mathcal E) be a measurable space, X:\Omega\times[0,\infty)\times E\to E be (\mathcal A\otimes\mathcal B([0,\infty))\otimes\... 0answers 61 views ### How does one define the gradient of a Markov semigroup? In the context of functional inequalities for Markov semigroups (\mathcal P_t)_{t\ge0}, what is one denoting by \nabla\mathcal P_tf? For example, I've found the following assumption in this paper: ... 1answer 92 views ### Existence of Markov chain on nonnegative integers with specified rates Let \lambda_k,\mu_k\in\mathbb R_{\ge0} (k\ge1) be nonnegative real numbers, let S=\mathbb Z_{\ge0} be the nonnegative integers, let T=\mathbb R_{\ge0} be the nonnegative real numbers and ... 1answer 267 views ### Calculate Radon-Nikodym derivative For the laws of two pure-jump Markov processes \mu_1 and \mu_2 on \mathbb R^n, which generators are H_1f(x)=\int h(x,dy) (f(y)-f(x)) and H_2f(x)=\int e^{-g(x,y)} h(x,dy) (f(y)-f(x)) (... 1answer 119 views ### Eigenspace of Gaussian Markov operator Consider the (one-dimensional) Gaussian distribution Q := N(\nu,\tau^2) and the (Gaussian) Markov operator \begin{equation*} \begin{array}{rccc} R : & L_1(\mathbb{R},\mathcal{B}(\mathbb{R}),Q) &... 2answers 172 views ### is this process a Markov one? Here is the problem I can't solve. Let \xi_n (n=1,2,3,\dots) be a sequence of i.i.d. random variables on \mathbb{R} with density p(x)>0, let \eta_n=\sum_{i=1}^{n}\xi_i^2. Define$$\... 1answer 72 views ### Conditions for optimal stationary strategies in MDPs I have a specific Markov decision process (MDP) which is generated from a problem in another domain. What I would like to show is that under the limit of means criterion (no discounting) the optimal ... 0answers 18 views ### Calculate regime-switching correlation matrix without assumption on distribution There's rich literature on Markov regime-switching dynamic correlation matrix. But most results seem to assume a certain kind of distribution and use MLE/EM. For example, some sort of multivariate ... 2answers 538 views ### what is the number of paths returning to 0 on the hexagonal lattice I am looking for an estimation of the number of paths of length$n$going from 0 to 0 on the hexagonal (or honeycomb) lattice. I can find plenty on references on self avoiding paths, but I am looking ... 0answers 52 views ### Algebraic property of a transition matrix Consider the simple random walk on$\mathbb{Z}^2$. Given a finite$\Sigma \subset \mathbb{Z}^2$, one can induce the random walk on$\Sigma$: set$\tau_0 = 0$, and define recursively$\tau_{n+1} := \...
Say I have a non-negative square $n\times n$ irreducible stochastic matrix $A$ (i.e., each column sums to 1), for which the following holds: $$A_{ij} > 0 \iff A_{ji} > 0.$$ I know that no more ...
Consider the infinitesimal generator $G$ of a Markov chain with state space $\mathbb{Z}$ such that it is symmetric with respect to a measure $\mu$ on $\mathbb{Z}$. Then, the operator \$(G,C_c(\mathbb{Z}...