# Questions tagged [markov-chains]

The tag has no usage guidance.

536 questions
Filter by
Sorted by
Tagged with
16 views

### Proof that Component-wise MH algorithm is invariant w.r.t. target measure

consider a standard situation in Bayesian modelling, given real vector parameter $\theta=(\theta_1,\dotsc,\theta_n)$ and observations $x$ we derive a posterior distribution $\pi$ with posterior ...
• 31
1 vote
79 views

230 views

### How to play golf in one dimension?

One-dimensional golf is a function $g$ on $\mathbb R$ such that $g(x)= 1+\min_\mu E[g(x+N(\mu,c\mu^2))]$ if $|x|>1$ and 0 if $|x|\le 1.$ Here $N$ is the normal distribution, whose mean $\mu$ you ...
• 18.2k
2k views

• 462
1 vote
51 views

50 views

### Multi-type Galton-Watson-like process where only majority-type is allowed to reproduce

Are you aware of any research papers that have explored a multi-type Galton-Watson process in which only particles of the majority type are permitted to reproduce in each generation? I've been unable ...
• 56
1 vote
108 views

We are given a queue of $n$ people $\{p_1, \ldots, p_n\}$. They each have to pass $n$ exams $\{t_1, \ldots, t_n\}$. For simplicity we can "draw" the setting in the following way: $$[t_n,t_{n-... • 21 1 vote 0 answers 70 views ### Bounding expectation of switching stochastic process I am analyzing the behavior of an 1D stochastic dynamic system, where the state can vary randomly within a small magnitude. However, when the state deviates too much from zero, its expected magnitude ... • 11 2 votes 0 answers 204 views ### Ball games: How to allocate N balls into M boxes so as to maximize the expected number of taken balls Consider the following ball games, which looks like very intuitive and simple but I have tried for a long time. Assuming we have M identical boxes and N identical balls, we distribute these N ... • 21 1 vote 0 answers 47 views ### Convergence of random variables based on shifts of a markov chain Suppose we have a discrete time (not necessarily stationary) Markov chain X=(X_0,X_1,X_2,\dots) on (\Omega, F). We assume X is Harris ergodic with an invariant distribution. Suppose we have a ... • 11 3 votes 1 answer 149 views ### Quantitative version of ergodic theorem in Markov chains Consider an irreducible Markov chain X_t with finite state space E, and unique invariant measure \pi. Fix a function V:E\to\mathbb R such that E_\pi[V]=0. The ergodic theorem tells us that, ... • 59 9 votes 2 answers 682 views ### Direct proof of unique invariant distribution for ergodic, positive-recurrent Markov chain I posted the following question on MSE, feeling that it perhaps isn't research level mathematics, but didn't get any bites. So, I am crossposting here. The following ergodic theorem is well known. ... • 165 1 vote 0 answers 35 views ### On a generator of a continuous-time Markov chain Let S be a countable set with discrete topology and let X=(\{X_t\}_{t \ge 0},\{P_x\}_{x \in S}) be a continuous-time Markov chain on S. We assume that each x \in S is a exponential holding ... • 701 2 votes 0 answers 111 views ### Can a diffusion process admit an invariant measure with a non-differentiable density? The precise domain of the generator A of an Itō diffusion on a Hilbert space H (assume H=\mathbb R^d, if that's easier for you to work with) can usually not be determined explicitly^1. Usually,... • 141 4 votes 1 answer 85 views ### The canonical path method for continuous-time Markov chains on a countable state space I wonder if anyone knows a research article that uses the canonical path method (or congestion ratio) to show that a Poincare inequality holds (see the images below) for a continuous-time Markov chain ... 3 votes 1 answer 215 views ### "Ergodic theorem" for Markov kernels Consider a discrete time Markov chain (X_t) on a finite state space \mathcal{S}, with transition matrix P. Assume that the chain admits a stationary distribution \pi, which I will identify ... -1 votes 1 answer 47 views ### Markov chain to solve a particle fusion problem A sequence of elementary particles arrive at Poisson rate r to a system. A pair of elementary particles can be fused into a level-1 particle. The fusion process succeeds with probability p_0. ... • 459 0 votes 0 answers 32 views ### How to find lower bounds of a modified mixing time (defined below) with respect to spectral of a finite Markov chain? I am focused on a time-homogeneous continuous-time Markov chain with a finite state space \mathcal{X}, whose Markov kernel is K and the corresponding semigroup is H_t=e^{-t(I-K)}. The invariant ... 1 vote 1 answer 92 views ### Asymptotic behavior of a Markov process on the set of \{0,1\}-polynomials This question is cross-posted from https://math.stackexchange.com/questions/4711799/asymptotic-behavior-of-a-markov-process-on-the-set-of-0-1-polynomials I am trying to study the asymptotic behavior ... 1 vote 0 answers 93 views ### Concatenation of Markov processes and independence In chapter 14 of Sharpe's General Theory of Markov Processes the concatenation of Markov processes X^1 and X^2 is described. I've posed the relevant part at the bottom of this post. It is rather ... • 141 1 vote 0 answers 64 views ### Reference for the asymptotic mixing time of the random walk on the cycle In Diaconis's book Group Representations in Probability and Statistics, Chapter 3C, there are explicit computations for the mixing time of the random walk on the cycle graph \mathbb{Z}_{p}, with p ... • 111 1 vote 0 answers 70 views ### Time-inhomogeneous Krylov-Bogoliubov Existence Theorem I am interested in what is known about the application of the Krylov-Bogoliubov existence theorem to the time-inhomogeneous case, especially as it relates to an underlying random dynamical system (... 3 votes 1 answer 98 views ### Comparison of time until absorption for two absorbing Markov chains Let \{X_t, t \geq 0\} and \{X_t', t \geq 0\} denote two markov chains on the same state space \{1, ..., n+1\} with transition probability matrices P and P' respectively. Suppose that both ... 0 votes 0 answers 73 views ### Convergence bounds for ergodic random walk We are given a simple connected graph G(V,E), where V and E denote the vertex and edge sets respectively. Let G'(V,E') be the graph generated by G by adding one self-loop edge for each ... • 1,667 1 vote 0 answers 40 views ### Langevin dynamics or stochastic gradient flow for grand canonical ensemble We know that for a measure exp(-U(X)) (canonical ensemble), we can use the dynamic dX=-DU(X)+ noise to sample the measure as t goes to infinity. Is there any dynamic corresponding to the grand ... 0 votes 0 answers 50 views ### About cutoff for quasi-random graphs In this paper by Hermon, Sly and Sousi about mixing time of a random walk on a random graph, there is a concept of \textit{regeneration edges} which I'm trying to understand. This is defined in page ... • 201 3 votes 1 answer 332 views ### Spectral Radius and Spectral Norm for Markov Operators My question concerns differences between the spectral radius \rho and norm \| \cdot \| of Markov operators in infinite-dimensional Banach spaces. This is far from my area of expertise, that is ... • 550 1 vote 0 answers 81 views ### Derive a closed-form expression of this recursive formula$$$$S(r,k) = f(r)S(0,k-1) + g(r)S(r+1,k-1)$$\ ,$$where r=0,1,2,\dots and k=1,2,3,\dots . Also, 0<f(r)<1 is an increasing function and 0<g(r)<1 is a ... 0 votes 0 answers 31 views ### Prove the explicit form of the ratio function in a Markov Chain Let (A_M^{\mathbb{Z}_+}, \Omega, P, \lambda) be a Markov Shift where A is a finite alphabet set, M is the admissibility matrix, P = [P_{i, j}]_{i, j\in A} is a stochastic matrix that is ... 2 votes 1 answer 215 views ### When is a stationary measure of a Markov chain "exponentially localized"? Here exponentially localized can be thought in a non-rigorous manner as a measure that is mostly supported on a sparse number of nodes. Some intuition can gained by thinking about a diffusion process, ... • 2,196 0 votes 1 answer 46 views ### Diameter of the range of composition of random maps on the circle My questions are related to the paper https://hal.science/hal-03933493v1 (accepted with corrections in Ergodic Theory and Dynamical Systems). I fix an irrational number \theta \in [0,1[. I define ... • 3,529 0 votes 0 answers 99 views ### On the exponentiation of a stochastic matrix where the exponent is a function of matrix size In this question, I asked about any arbitrary stochastic matrix A(n) of the particular form$$A(n) = \begin{pmatrix} 1 & 0 & \cdots & 0\\ x_{21} & x_{22} & \cdots & 0\\ \...
I wonder whether a $d$-dimensional random walk $S_n$, generated by the infinite i.i.d. copies of X given by: $X=e_1=(1, 0, 0, ..., 0)$ (with probability $p_1$) $X=e_2=(0, 1, 0, ..., 0)$ (with ...