# Tagged Questions

The markov-chains tag has no usage guidance.

**6**

votes

**2**answers

573 views

### References for a physicist migrating to stochastic processes

I've studied "Markov Chains" - Norris and "Measure, Integral and Probability" - Capinski, Kopp. Now, I'm looking for a couple of books (or other references) that help me bridging these two topics. ...

**3**

votes

**0**answers

102 views

### Mixing time for dimers on the square-octagon graph

Consider the "fortress graph" of order $n$ (see Figure 9 of http://faculty.uml.edu/jpropp/tiling/www/mdblum/arctic.html). It's been known empirically for twenty years that if one turns the set of ...

**6**

votes

**3**answers

5k views

### Time-inhomogeneous Markov Chains

I'm trying to find out what is known about time-inhomogeneous ergodic Markov Chains where the transition matrix can vary over time. All textbooks and lecture notes I could find initially introduce ...

**0**

votes

**1**answer

128 views

### Monte Carlo estimator with autocorrelated samples

Given an integration problem $I=\int{f(x)dx}$, we can construct an ordinary Monte Carlo estimator as
$E[I]=\sum\limits_i\frac{f(x_i)}{p(x_i)}$
where the samples $x_i$ are usually i.i.d. and drawn ...

**4**

votes

**1**answer

195 views

### Local Markov implies global Markov

Let $G=(V,E)$ be a finite simple graph, and let $\{X_i\}_{i \in V}$ be a collection of random variables associated with the vertices of $G$. The joint distributions of these r.v.s is a Markov Random ...

**0**

votes

**1**answer

195 views

### Is any invariant, ergodic measure with full support on an irreducible Markov shift a Markov measure?

I have this question I have been struggling with for a while. It seems rather intuitive, however, I was not able to proof it yet:
Let $\Omega = \{1,2,\cdots,N\}$ a finite alphabet, $\Sigma \subset \...

**4**

votes

**3**answers

376 views

### Invariant measure of Euler-Maruyama Discretisation of an Ito diffusion

Let $(X_t)_{t \geq 0}$ be a diffusion process with dynamics governed by the stochastic differential equation
\begin{equation}
dX_t = b(X_t)dt + \sigma(X_t)dW_t, ~~ X_0 = x_0,
\end{equation}
where $b,\...

**2**

votes

**1**answer

216 views

### convergence rate of occupation measure of ergodic Markov Chain

Given an ergodic Markov chain $(X_n)_{n\geq 1}$ in $R^d$with $\pi$ as the invariant distribution of the transition kernel, under good conditions we have that the empirical occupation measure converges ...

**1**

vote

**1**answer

252 views

### forward algorithm Hidden Markov Model

I am studying the the forward-backward algorithm used in Hidden Markov Models. I understand that that you are trying to propagate through a sequence (and the available states) to find the most ...

**3**

votes

**0**answers

115 views

### Nonlinear Markov process

Consider the following nonlinear $\mathbb{R}$-valued stochastic recursive sequence:
$ X_{n+1} = F(X_n) + W_{n+1}, \quad (W_n)_{n\ge1} \stackrel{ \scriptsize \mathrm{i.i.d.} }{ \sim } \phi. $
How can ...

**3**

votes

**2**answers

302 views

### Markov Partitions for toral automorphisms

I know that my question is more practical than theoretical. But, I do know where to look for the theoretical sources.
I want to find a program in the case that it exists (does it?), or to program it. ...

**1**

vote

**0**answers

289 views

### segmental k-means hmm algorithm

I'm trying to determine the state parameters for an hmm with a minimum of 9 states. I'm running chunks of my data through hmmtrain on MATLAB which I think uses the Baum/Welch aka forward/backward ...

**2**

votes

**0**answers

781 views

### Closed-form solution to a system of linear equations

Consider the following $n \times n$ matrix with a particularly nice structure:
\begin{equation}\mathbf{P}=\begin{pmatrix}
0 & 0& \dots&0 & 0 &1\\
0 & 0& \dots&0 & \...

**0**

votes

**1**answer

347 views

### Generating independent random variable from two correlated random variables

Suppose two random variables $X$ and $V$ are given. I am wondering what kind of condition we need to impose on joint distribution of $V$ and $X$ to make sure that there exists a random variable $Z$ ...

**14**

votes

**4**answers

985 views

### Eigenvectors of a particular transition matrix

I am considering a Markov chain with $n$ states with a particularly nice structure. The transition matrix is as follows:
\begin{equation}\mathbf{P}=\begin{pmatrix}
0 & 0& \dots&0 & 0 &...

**1**

vote

**0**answers

83 views

### CLT for a Markov Renewal Process

Suppose $(X,T)=\{(X_n,T_n)\}_{n\geq0}$ is a Markov renewal process, where $X$ is a finite-state, discrete-time Markov chain with state space $\{1,2,...,R\}$. $T$ is the additive component, more ...

**6**

votes

**1**answer

321 views

### Quasi-stationary distribution for a death process

In the paper, Survival in a quasi-death process by van Doorn and Pollett, the quasi-stationary distribution of a transient CTMC is discussed and QSD for a simple death process is derived.
Consider a ...

**2**

votes

**1**answer

248 views

### Transition probabilities in coupled Markov chains

I know that for a continuous-time Markov chain, the probability of transition from time $0$ to $t$ is given by $P(t)=e^{Q(t)t}$. I have a system of $N$ interdependent continuous-time Markov chains ...

**1**

vote

**1**answer

89 views

### Minimal variance for phase-type distributions?

Let $\mathcal{D}(m)$ be the set of phase-type distributions constructed from $m+1$-state Markov chains. Recall that the coefficient of variation of a distribution $D$ is the ratio of the standard ...

**3**

votes

**0**answers

317 views

### Maximization of a total variation distance subject to another total variation distance in Markov chain

Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the ...

**5**

votes

**1**answer

135 views

### Relative vulnerabilities in SIS epidemic model

Consider the SIS model of epidemic spreading. There is a finite graph $G(V,E)$, link infection rates $\lambda_{ij}$ and node recovery rates $\mu_i$. There are a few initial nodes which are infected at ...

**4**

votes

**1**answer

160 views

### Log-likelihood in regime switching

Not sure if this question is too simple to be asked here...
In the following paper
Cho, Jin Seo, and Halbert White. "Testing for regime switching." Econometrica 75.6 (2007): 1671-1720. doi: 10....

**6**

votes

**1**answer

190 views

### A family of skew-symmetric matrices corresponding to cycles in graphs

When investigating loops in Markov chains I ran into the following observation.
A cycle in a graph $G$ with $n$ vertices may be represented by a matrix $\Gamma \in \mathbb R^{n \times n}$ having the ...

**2**

votes

**0**answers

147 views

### An optimization in Markov Chain

We are given two correlated random variables $V$ and $X$ supported over a finite alphabets $\mathcal{V}$ and $\mathcal{X}$. Suppose the marginal $P_V$ and conditional distribution $P_{X|V}$ are ...

**0**

votes

**1**answer

733 views

### Stationary distribution in general Markov Chains

This is just a reference request for a result which is very general, useful and should be well-known, but I've failed to find a good reference to cite.
The problem is to define the "most natural" ...

**6**

votes

**1**answer

185 views

### Finding cohesive (low exit probability) sets in a Markov process

The following is a fact about Markov chains that came up in a game theory paper. The purpose of this question is to ask if related notions or similar results are found elsewhere in probability, or are ...

**0**

votes

**0**answers

126 views

### Is it unique when a irreducible and aperiodic markov chain on general space has an invariant measure?

Recently I'm reading Markov chains and Stochastics Stability(sencond edition 2009) written by Meyn and Tweedie. And in the proof of Theorem 10.4.5 on page 243, it says "if $\pi_m$ is invariant for the ...

**0**

votes

**1**answer

442 views

### Supremum in a Markov chain model

A Markov chain $X$ with finite state space $\{1,2,\cdots,N\}$ is defined on a probability space $(\Omega, P, \mathcal{F})$ equiped with filtration $\{\mathcal{F}_t\}$. And we assume that we can reach ...

**3**

votes

**4**answers

388 views

### Does the variance of a continuous time, time homogeneous, Markov process starting from one point necessarily not decrease?

Let $x_t$ be a zero mean, time homogeneous Markovian process (chiefly look at the case where the value is in $1$ dimension) over time $t$ starting from $x_0=0$. Is it necessary that, in continuous ...

**1**

vote

**0**answers

146 views

### The problem of the drunkard in a valley [closed]

We consider a Markov chain on a subset of positive integers S = {0, 1, 2, 3, .......N}, with transition probabilities defined as follows:
The chain jumps only one unit to the left or right.
p(i, j) =...

**4**

votes

**2**answers

255 views

### Anticoncentration of the convolution of two characteristic functions

Edit: This is a question related to my other post, stated in a much more concrete way I think.
I am interested in anything (ideas, references) related to the following problem:
Suppose that $A \...

**3**

votes

**0**answers

87 views

### Best convergence rate for convolutions on $\mathbb{Z}_p$

Suppose, that we have sequence of i.i.d variables $X_1,\ldots,X_n$ taking values in $\mathbb{Z}_p$ such that $d_{TV}(X_1,U) < \delta$.
How fast, in terms of $\delta$ and $n$ does the sum $X_1+\...

**7**

votes

**2**answers

348 views

### Convergence rate of the convolution of almost uniform measures on $\mathbb{Z}_p$

Statement Given a finite abelian group $G$ and two independent random variables $X,Y$ taking values in $G$ and satisfying $d_{TV}(X,U_G)\leqslant \delta$ and $d_{TV}(Y,U_G)\leqslant \delta$ (where $...

**4**

votes

**0**answers

211 views

### Generalized Markov Processes on CW complexes of dimension > 1

Markov processes have a large variety of applications to physics and chemistry (as well as many other fields). Such processes are formulated on graphs, i.e., CW complexes of dimension one. It is ...

**0**

votes

**3**answers

1k views

### Stationary distribution for bipartite graph

I was wondering if there is any stationary distribution for bipartite graph? Can we apply random walks on bipartite graph? since we know the stationary distribution can be found from Markov chain, but ...

**2**

votes

**0**answers

203 views

### Kullback-Leibler Divergence of Stationary Distributions of Markov chains

Consider two finite Markov chains on the same state space, both assumed to be irreducible, with transition matrices $P$ and $Q$ and associated stationary distributions $\pi$ and $\tilde \pi$. Is it ...

**1**

vote

**1**answer

143 views

### Optimum control of a probabilistic automaton

Suppose we have a probabilistic automaton and we assign a weight to each state. An "interaction strategy" would be a fixed map from states to inputs. Any interaction strategy could be used to ...

**5**

votes

**0**answers

624 views

### Coin Toss Probabilities like Penney's Game

Generate a binary number, using coin toss. Until you receive a predefined terminating sequence. What is the probability that the number is a multiple of some $k$.
For example, the terminating ...

**4**

votes

**1**answer

389 views

### Uniqueness of Gibbs Measure on Ising model

If I understood this correctly, the Gibbs Specification for the Ising model on $ℤ^d$ dos not have a unique Gibbs Measure for β above the critical level. But what about the Ising model on a finite ...

**4**

votes

**2**answers

217 views

### Regarding Ricci curvature of Markov chains

In Ricci curvature of Markov chains on metric spaces Yann Ollivier, defines a coarse Ricci curvature for a Markov chain with transition kernels $\{m_x\}$ defined on a metric space $(X,d)$ as follows: ...

**0**

votes

**1**answer

481 views

### Markov Chain: state reduction

Hi I am trying to understand a proof in a paper (written by Isaac Sonin), I don't know if anyone could give me a clarification on the following:
Firstly we have a Markov chain $\{Y_k\}$ with finite ...

**3**

votes

**0**answers

188 views

### Markov operators and existence of ergodic measures

My question refers to the yesterday's question (see here)
of John Learner and goes as follows:
Can we deduce the existence of an ergodic measure if we know that an invariant measure exists, but the ...

**2**

votes

**0**answers

65 views

### iterated mutinomial and hitting time

Let $N,k \geq 1$ be two integers and consider the following Markov chain on $[0,k\times N]^k$. It starts at $X^0=(N, N, \ldots, N)$ and $X^{n+1}$ is the realisation of a multinomial distribution with $...

**0**

votes

**0**answers

118 views

### Inequality relating stationary probabilities and transition probabilities

Let $P$ be the transition probability matrix of a aperiodic irreducible DTMC and let $\pi$ be its stationary distribution. I would like to know if there is any literature on types of Markov chains ...

**8**

votes

**0**answers

113 views

### Functions between Markov chains that preserve local harmonicity

Given two Markov chains with respective state-spaces $S$ and $T$, say that a function $\phi$ from $S$ to $T$ is holomorphic iff for all states $t \in T$, every real-valued function $f$ on $T$ that is ...

**3**

votes

**1**answer

152 views

### Markov Chains based on sampled transition probabilities [closed]

If I have a process that transitions between states with some set, unknown probability, I can sample to find the transition probability. This probability is a sample average, with a well understood ...

**1**

vote

**1**answer

160 views

### Is anything known about Large Deviation Principle for non additive functionals on Markov chains?

Let $\Sigma$ be a finite set of cardinality $|\Sigma |$ and
$$\Pi = \{ \pi(i,j)\}_{i,j = 1}^{|\Sigma|}$$
a stochastic matrix (ie a matrix whose elements are non negative and such that
each row sum ...

**0**

votes

**1**answer

210 views

### A basic question on necessary and sufficient condition for positive recurrence

If state $j$ is recurrent and the following holds can it be called as positive recurrent ?
$$\lim_{n -> \infty}\frac{1}{n}\sum_{k=1}^{n}p_{jj}^{(k)} > 0$$
I know that this a necessary ...

**4**

votes

**0**answers

145 views

### Relaxation = absorption?

Let $A$ be a stochastic matrix, that is, the entries are non-negative and each row adds to $1$. Assume that it is primitive, that is, $A^n$ has only positive entries for sufficiently large $n$. We ...

**2**

votes

**1**answer

243 views

### The first eigenvalue of a branching process matrix

Let $M$ be the real square matrix of a typed branching process, such that $M_{ij}$ is the expected value of offspring of type $j$ emanating from type $i$.
We know that if the first eigenvalue if $M$ ...