The markov-chains tag has no usage guidance.

**1**

vote

**1**answer

248 views

### forward algorithm Hidden Markov Model

I am studying the the forward-backward algorithm used in Hidden Markov Models. I understand that that you are trying to propagate through a sequence (and the available states) to find the most ...

**3**

votes

**0**answers

114 views

### Nonlinear Markov process

Consider the following nonlinear $\mathbb{R}$-valued stochastic recursive sequence:
$ X_{n+1} = F(X_n) + W_{n+1}, \quad (W_n)_{n\ge1} \stackrel{ \scriptsize \mathrm{i.i.d.} }{ \sim } \phi. $
How can ...

**3**

votes

**2**answers

299 views

### Markov Partitions for toral automorphisms

I know that my question is more practical than theoretical. But, I do know where to look for the theoretical sources.
I want to find a program in the case that it exists (does it?), or to program it. ...

**1**

vote

**0**answers

276 views

### segmental k-means hmm algorithm

I'm trying to determine the state parameters for an hmm with a minimum of 9 states. I'm running chunks of my data through hmmtrain on MATLAB which I think uses the Baum/Welch aka forward/backward ...

**2**

votes

**0**answers

768 views

### Closed-form solution to a system of linear equations

Consider the following $n \times n$ matrix with a particularly nice structure:
\begin{equation}\mathbf{P}=\begin{pmatrix}
0 & 0& \dots&0 & 0 &1\\
0 & 0& \dots&0 & \...

**0**

votes

**1**answer

347 views

### Generating independent random variable from two correlated random variables

Suppose two random variables $X$ and $V$ are given. I am wondering what kind of condition we need to impose on joint distribution of $V$ and $X$ to make sure that there exists a random variable $Z$ ...

**14**

votes

**4**answers

979 views

### Eigenvectors of a particular transition matrix

I am considering a Markov chain with $n$ states with a particularly nice structure. The transition matrix is as follows:
\begin{equation}\mathbf{P}=\begin{pmatrix}
0 & 0& \dots&0 & 0 &...

**1**

vote

**0**answers

83 views

### CLT for a Markov Renewal Process

Suppose $(X,T)=\{(X_n,T_n)\}_{n\geq0}$ is a Markov renewal process, where $X$ is a finite-state, discrete-time Markov chain with state space $\{1,2,...,R\}$. $T$ is the additive component, more ...

**6**

votes

**1**answer

320 views

### Quasi-stationary distribution for a death process

In the paper, Survival in a quasi-death process by van Doorn and Pollett, the quasi-stationary distribution of a transient CTMC is discussed and QSD for a simple death process is derived.
Consider a ...

**2**

votes

**1**answer

227 views

### Transition probabilities in coupled Markov chains

I know that for a continuous-time Markov chain, the probability of transition from time $0$ to $t$ is given by $P(t)=e^{Q(t)t}$. I have a system of $N$ interdependent continuous-time Markov chains ...

**1**

vote

**1**answer

89 views

### Minimal variance for phase-type distributions?

Let $\mathcal{D}(m)$ be the set of phase-type distributions constructed from $m+1$-state Markov chains. Recall that the coefficient of variation of a distribution $D$ is the ratio of the standard ...

**3**

votes

**0**answers

288 views

### Maximization of a total variation distance subject to another total variation distance in Markov chain

Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the ...

**5**

votes

**1**answer

135 views

### Relative vulnerabilities in SIS epidemic model

Consider the SIS model of epidemic spreading. There is a finite graph $G(V,E)$, link infection rates $\lambda_{ij}$ and node recovery rates $\mu_i$. There are a few initial nodes which are infected at ...

**4**

votes

**1**answer

160 views

### Log-likelihood in regime switching

Not sure if this question is too simple to be asked here...
In the following paper
Cho, Jin Seo, and Halbert White. "Testing for regime switching." Econometrica 75.6 (2007): 1671-1720. doi: 10....

**6**

votes

**1**answer

190 views

### A family of skew-symmetric matrices corresponding to cycles in graphs

When investigating loops in Markov chains I ran into the following observation.
A cycle in a graph $G$ with $n$ vertices may be represented by a matrix $\Gamma \in \mathbb R^{n \times n}$ having the ...

**2**

votes

**0**answers

144 views

### An optimization in Markov Chain

We are given two correlated random variables $V$ and $X$ supported over a finite alphabets $\mathcal{V}$ and $\mathcal{X}$. Suppose the marginal $P_V$ and conditional distribution $P_{X|V}$ are ...

**0**

votes

**1**answer

720 views

### Stationary distribution in general Markov Chains

This is just a reference request for a result which is very general, useful and should be well-known, but I've failed to find a good reference to cite.
The problem is to define the "most natural" ...

**6**

votes

**1**answer

185 views

### Finding cohesive (low exit probability) sets in a Markov process

The following is a fact about Markov chains that came up in a game theory paper. The purpose of this question is to ask if related notions or similar results are found elsewhere in probability, or are ...

**0**

votes

**0**answers

126 views

### Is it unique when a irreducible and aperiodic markov chain on general space has an invariant measure?

Recently I'm reading Markov chains and Stochastics Stability(sencond edition 2009) written by Meyn and Tweedie. And in the proof of Theorem 10.4.5 on page 243, it says "if $\pi_m$ is invariant for the ...

**0**

votes

**1**answer

422 views

### Supremum in a Markov chain model

A Markov chain $X$ with finite state space $\{1,2,\cdots,N\}$ is defined on a probability space $(\Omega, P, \mathcal{F})$ equiped with filtration $\{\mathcal{F}_t\}$. And we assume that we can reach ...

**3**

votes

**4**answers

388 views

### Does the variance of a continuous time, time homogeneous, Markov process starting from one point necessarily not decrease?

Let $x_t$ be a zero mean, time homogeneous Markovian process (chiefly look at the case where the value is in $1$ dimension) over time $t$ starting from $x_0=0$. Is it necessary that, in continuous ...

**1**

vote

**0**answers

145 views

### The problem of the drunkard in a valley [closed]

We consider a Markov chain on a subset of positive integers S = {0, 1, 2, 3, .......N}, with transition probabilities defined as follows:
The chain jumps only one unit to the left or right.
p(i, j) =...

**4**

votes

**2**answers

252 views

### Anticoncentration of the convolution of two characteristic functions

Edit: This is a question related to my other post, stated in a much more concrete way I think.
I am interested in anything (ideas, references) related to the following problem:
Suppose that $A \...

**3**

votes

**0**answers

87 views

### Best convergence rate for convolutions on $\mathbb{Z}_p$

Suppose, that we have sequence of i.i.d variables $X_1,\ldots,X_n$ taking values in $\mathbb{Z}_p$ such that $d_{TV}(X_1,U) < \delta$.
How fast, in terms of $\delta$ and $n$ does the sum $X_1+\...

**7**

votes

**2**answers

346 views

### Convergence rate of the convolution of almost uniform measures on $\mathbb{Z}_p$

Statement Given a finite abelian group $G$ and two independent random variables $X,Y$ taking values in $G$ and satisfying $d_{TV}(X,U_G)\leqslant \delta$ and $d_{TV}(Y,U_G)\leqslant \delta$ (where $...

**4**

votes

**0**answers

210 views

### Generalized Markov Processes on CW complexes of dimension > 1

Markov processes have a large variety of applications to physics and chemistry (as well as many other fields). Such processes are formulated on graphs, i.e., CW complexes of dimension one. It is ...

**0**

votes

**3**answers

1k views

### Stationary distribution for bipartite graph

I was wondering if there is any stationary distribution for bipartite graph? Can we apply random walks on bipartite graph? since we know the stationary distribution can be found from Markov chain, but ...

**2**

votes

**0**answers

195 views

### Kullback-Leibler Divergence of Stationary Distributions of Markov chains

Consider two finite Markov chains on the same state space, both assumed to be irreducible, with transition matrices $P$ and $Q$ and associated stationary distributions $\pi$ and $\tilde \pi$. Is it ...

**1**

vote

**1**answer

143 views

### Optimum control of a probabilistic automaton

Suppose we have a probabilistic automaton and we assign a weight to each state. An "interaction strategy" would be a fixed map from states to inputs. Any interaction strategy could be used to ...

**4**

votes

**0**answers

568 views

### Coin Toss Probabilities like Penney's Game

Generate a binary number, using coin toss. Until you receive a predefined sequence. What is the probability that the number is a multiple of some k.
For example, the terminating sequence could be '11'...

**4**

votes

**1**answer

386 views

### Uniqueness of Gibbs Measure on Ising model

If I understood this correctly, the Gibbs Specification for the Ising model on $ℤ^d$ dos not have a unique Gibbs Measure for β above the critical level. But what about the Ising model on a finite ...

**4**

votes

**2**answers

215 views

### Regarding Ricci curvature of Markov chains

In Ricci curvature of Markov chains on metric spaces Yann Ollivier, defines a coarse Ricci curvature for a Markov chain with transition kernels $\{m_x\}$ defined on a metric space $(X,d)$ as follows: ...

**0**

votes

**1**answer

473 views

### Markov Chain: state reduction

Hi I am trying to understand a proof in a paper (written by Isaac Sonin), I don't know if anyone could give me a clarification on the following:
Firstly we have a Markov chain $\{Y_k\}$ with finite ...

**3**

votes

**0**answers

186 views

### Markov operators and existence of ergodic measures

My question refers to the yesterday's question (see here)
of John Learner and goes as follows:
Can we deduce the existence of an ergodic measure if we know that an invariant measure exists, but the ...

**2**

votes

**0**answers

65 views

### iterated mutinomial and hitting time

Let $N,k \geq 1$ be two integers and consider the following Markov chain on $[0,k\times N]^k$. It starts at $X^0=(N, N, \ldots, N)$ and $X^{n+1}$ is the realisation of a multinomial distribution with $...

**0**

votes

**0**answers

118 views

### Inequality relating stationary probabilities and transition probabilities

Let $P$ be the transition probability matrix of a aperiodic irreducible DTMC and let $\pi$ be its stationary distribution. I would like to know if there is any literature on types of Markov chains ...

**8**

votes

**0**answers

113 views

### Functions between Markov chains that preserve local harmonicity

Given two Markov chains with respective state-spaces $S$ and $T$, say that a function $\phi$ from $S$ to $T$ is holomorphic iff for all states $t \in T$, every real-valued function $f$ on $T$ that is ...

**3**

votes

**1**answer

151 views

### Markov Chains based on sampled transition probabilities [closed]

If I have a process that transitions between states with some set, unknown probability, I can sample to find the transition probability. This probability is a sample average, with a well understood ...

**1**

vote

**1**answer

160 views

### Is anything known about Large Deviation Principle for non additive functionals on Markov chains?

Let $\Sigma$ be a finite set of cardinality $|\Sigma |$ and
$$\Pi = \{ \pi(i,j)\}_{i,j = 1}^{|\Sigma|}$$
a stochastic matrix (ie a matrix whose elements are non negative and such that
each row sum ...

**0**

votes

**1**answer

208 views

### A basic question on necessary and sufficient condition for positive recurrence

If state $j$ is recurrent and the following holds can it be called as positive recurrent ?
$$\lim_{n -> \infty}\frac{1}{n}\sum_{k=1}^{n}p_{jj}^{(k)} > 0$$
I know that this a necessary ...

**4**

votes

**0**answers

144 views

### Relaxation = absorption?

Let $A$ be a stochastic matrix, that is, the entries are non-negative and each row adds to $1$. Assume that it is primitive, that is, $A^n$ has only positive entries for sufficiently large $n$. We ...

**2**

votes

**1**answer

241 views

### The first eigenvalue of a branching process matrix

Let $M$ be the real square matrix of a typed branching process, such that $M_{ij}$ is the expected value of offspring of type $j$ emanating from type $i$.
We know that if the first eigenvalue if $M$ ...

**1**

vote

**0**answers

206 views

### Stationary distribution of a countable state Markov chain

We assume the Markov chain to be countable state space, time-homogeneous. Does it necessarily have a stationary distribution? I found a paper on arXiv.org (http://arxiv.org/abs/math/0610707) that ...

**1**

vote

**1**answer

120 views

### Empirical distribution of a collection of iid Markov chains

Suppose we have $N$ independent 2-point Markov chains each having a rate matrix $Q = [-1,1;1,-1]$ and stationary distribution $\pi = [0.5,0.5]$. At time $t=0$, we initiate the chains so that the ...

**0**

votes

**0**answers

100 views

### when are two Markov chains same distributions

Let there be two Markov processes on the same state space (which is countably infinite), but different transition matrices, denoted by $P_{1}$ and $P_{2}$. Assume positive recurrence, irreducibility ...

**5**

votes

**1**answer

1k views

### Efficient computation of Markov chain transition probability matrix

Consider a continuous Markov chain $X = (X_t)$ on a finite state space and let $Q$ be the (given) transition rate matrix. This matrix is very sparse, with non-zero values on 3 diagonals only (so from ...

**2**

votes

**1**answer

848 views

### Hitting time probability in a Random Walk with possibility to die.

A Random Walker can move of one unit to the right with probability $p$, to the left with probability $q$ and it can jump again to the starting point with probability $r$ and die. Naturally $p+q+r=1$. ...

**1**

vote

**1**answer

252 views

### Equivalent Markov Random Fields

Hi,
Is it possible to have topologically different Markov Random Fields (few different edges) and yet yielding the same inference results ?
Thanks!

**1**

vote

**2**answers

313 views

### The limiting behavior of geometric random walk

I would like to know what the asymptotic limiting behavior is for the following random walk on $\mathbb Z^d$. By Donsker's invariance principle, I suspect that its behavior is diffusive, i.e., the ...

**0**

votes

**0**answers

540 views

### Markov transition probabilities and negative binomial distribution.

A realization of a Markov process generates a sequence of interval lengths between transition from one state to another. A natural way of modeling the distribution of the lengths is as a negative ...