The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
14 views

Markov Modulated Markov Chain

Consider a discrete time Markov chain $X_t$ on some finite state space $\mathcal{S}$ with transition matrix $P$. Now consider a process $Y_t$ also on $\mathcal{S}$, which conditioned on $X_{t}=s$ ...
1
vote
0answers
33 views

Eigenvectors of a perturbed reducible stochastic matrix

Let $Q$ be a $n\times n$ reducible stochastic matrix. Let $J$ be such that $[J]_{ij}={1 \over n}$. Now for a small positive constant $\alpha\in [0,1]$, consider the matrix ...
3
votes
1answer
73 views

How much larger than the relaxation time can the mixing time be?

The notation is mostly taken from the book "Markov chains and mixing times" by Levin, Peres, and Wilmer. Consider an irreducible, aperiodic, time-reversible, discrete-time Markov chain on a finite ...
0
votes
0answers
51 views

Circular process ergodic?

Let us define a continuous-time Markov process on a circle consisting of $m-$ equally spaced points, i.e. every point has two neighbours. Now, we define a space of functions $S:= ...
-1
votes
1answer
78 views

Mixing time of lazy random walk on the directed cycle $C_n$

Briefly: A hint (if this is easy), reference or derivation would be of great help. The question Let $C_n$ be the directed cycle with loops in each of its $n$ vertices, and consider the random walk ...
0
votes
1answer
364 views

Supremum in a Markov chain model

A Markov chain $X$ with finite state space $\{1,2,\cdots,N\}$ is defined on a probability space $(\Omega, P, \mathcal{F})$ equiped with filtration $\{\mathcal{F}_t\}$. And we assume that we can reach ...
1
vote
0answers
14 views

Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models. Consider a hidden Markov model (HMM) with ...
0
votes
0answers
44 views

Regularity of the entrance measure of SRW

Let $S(n)$ be the discrete sphere of radius $n$ (i.e., the internal boundary of the Euclidean discrete ball $B(n)$) centered in the origin, and consider a simple random walk starting at some ...
4
votes
2answers
200 views

Regarding Ricci curvature of Markov chains

In Ricci curvature of Markov chains on metric spaces Yann Ollivier, defines a coarse Ricci curvature for a Markov chain with transition kernels $\{m_x\}$ defined on a metric space $(X,d)$ as follows: ...
5
votes
1answer
251 views

Deterministic finite-state automaton driven by a Markov chain

I've stumbled on some problem, and I have the feeling that this is closed to something well-studied in dynamical systems. The problem is the following. Consider a finite-state automaton with state ...
0
votes
0answers
50 views

Order statistic of Markov chain sample path and related probabilities

Consider a 1D sample path, denoted as $\{X(1), ..., X(t), ..., X(n)\}$, generated from a discrete time finite state (time homogeneous) Markov chain over states $\{1,...,m\}$, with transition ...
1
vote
0answers
22 views

steady state of a continuous-time birth-death process

we consider a continuous-time birth-death process $\{X(t),t\geq 0\}$ with discrete state space taking non0negative integer values $\{0,1,2,3,...\}$. The transition rates of the process $\{X(t)\}$ are ...
3
votes
1answer
55 views

Similarity transformation of transition matrix of reversible Markov chain (reference request)

If $P$ is the transition matrix of a reversible Markov chain, and $\pi$ is its stationary distribution, and let $R$ be defined by: $$R_{ij} = \sqrt{\frac{\pi_i}{\pi_j}}P_{ij}~.$$ By reversibility, ...
0
votes
0answers
24 views

steady state distribution of the following infinite-state Markov chain

Given the following state transition equation: $P_0(n+1)=P_0(n)(1-\lambda \Delta t)+ P_1(n)\mu \Delta t$ $P_j(n+1)=P_{j}(n)(1-\lambda \Delta t-\mu \Delta t)+\lambda \Delta t P_{j-1}(n)+ \mu \Delta ...
2
votes
0answers
34 views

Spectral gap of two step Markov chain

Suppose $X_1,X_2,\ldots$ is a reversible Markov chain with state space of size $k$ and absolute spectral gap $\gamma_*.$ What is the spectral gap of the (non-reversible) Markov chain $Y_1,Y_2,\ldots,$ ...
6
votes
1answer
314 views

Quasi-stationary distribution for a death process

In the paper, Survival in a quasi-death process by van Doorn and Pollett, the quasi-stationary distribution of a transient CTMC is discussed and QSD for a simple death process is derived. Consider a ...
0
votes
0answers
26 views

steady state distribution for a jump Markov chain

Consider a queueing process with the following transition matrix: $\mathbf{P}=\left( \begin{smallmatrix} 1-\lambda & \lambda & & & & & & &\\ \mu & ...
0
votes
1answer
79 views

Finite hitting time implies hits at any finite time?

I was wondering about the following problem: Assume we have a state space $S:=\mathbb{Z}$ and a Markov chain, such that we can go from any state $x$ to some state $y$ with positive probabilities, ...
0
votes
1answer
102 views

Markov chain with Feller property

Does anybody know whether there is an analysis of when the monotone decreasing chain has the Feller-property? The monotone decreasing is defined as a chain on $\mathbb{N}$ and the rate of going down ...
0
votes
0answers
33 views

Calculate the KL divergence between two transition matrices

I want to calculate how different two markov transition matrices are. For example: $\begin{pmatrix} .2 & .8 \\ .1 & .9 \end{pmatrix}$ and $\begin{pmatrix} .3 & .7 \\ .1 & .9 ...
3
votes
0answers
38 views

Memorylessness of residence times for a Markov process

I'm stuck on the trivial problem of showing memorylessness of holding (residence) times for a continuous time homogeneous Markov chain on finite state space. I have a homogeneous Markov process ...
8
votes
1answer
141 views

Equalizing Geometric means of Graph Cycles

Consider a strongly connected directed graph $G$. I have been stuck on the following question: can you assign real numbers in $[0,1]$ to each edge of $G$ so that the geometric mean of all cycles are ...
0
votes
0answers
35 views

positive Harris recurrent, aperiodic, stationary Markov chain

How to proof that every positive Harris recurrent, aperiodic, stationary Markov chain is alpha-mixing (strong-mixing)?
1
vote
1answer
119 views

Two types of random walkers on square lattice

Consider a two dimensional square lattice ($n$ by $n$), which is our space $S$ (each point labelled by an index $1\to n^2$), containing two types of particles, distinguished here by either an index ...
3
votes
1answer
100 views

Markov-semigroup sobolev inequality

I have a question about the following definition: A probability measure $\mu$, such that the Markov semi-group $e^{Lt} \in L(L^2)$ exists and is symmetric, satisfies the Sobolev inequality iff for ...
0
votes
0answers
30 views

maximum for a nonstationary Markov Chain

Let $\{X_j\}$ be a nonstationary Markov Chain with transition matrix $P$. What is a relation between $P(M_{1:n} \leq x)\quad \text{and}\quad P(M_{1:T} \leq x), \quad P(M_{T:n} \leq x)$ where $M_{k:n} ...
3
votes
0answers
99 views

The spring Markov chain on $\mathbb{N}$

I'm trying to understand and learn more about "almost surely bounded" Markov chains on countable state spaces. I'm looking for references where I can learn how to work with more complicated examples ...
1
vote
0answers
35 views

Is there an effective algorithm for finding “minimal discovery times” for large graphs?

Consider a large, probably sparse graph with Markovian random walkers on it. Define the discovery time as the expected time to first reach a vertex by random walk from a uniform start. Are there ...
0
votes
0answers
60 views

Markov chain matching local time

Let $\left(X_{t}\right)_{t\geq0}$ be a Markov process taking values in a finite state space $E$. Its local time at $y\in E$ started at $x\in E$ is defined as $$ ...
1
vote
1answer
63 views

General Markov Chains on same Probability Space?

A Markov chain $(X_i)_{i\in \mathbb{N}}$ on a measurable space $(E,\Sigma)$ is (see e.g. Revuz or Meyn/Tweedie) constructed on the following probabilty space. $$ \Omega = \{ (x_l)_{l \in \mathbb{N}} ...
7
votes
1answer
250 views

Existence of Limiting Distribution for Moving Regions in Stat. Phys. Models

As the title (hopefully) suggests, I've been trying to prove (or disprove) the existence of a limiting distribution for a certain projection in a statistical physics model. I'll give the details of ...
0
votes
0answers
37 views

Validating a probability density distribution forecast model for a Markov process

Let's say we have a Markov process $X_t$, and we come up with a forecast model that takes some information from outside world and says: "value $X_{t+1}$ has probability density distribution $P_t(x)$". ...
31
votes
6answers
3k views

Deep Learning / Deep neural nets for mathematician

I am interested in finding out the math ideas behind the technologies that are under the umbrella of "Deep Learning" or "Deep neural nets". Most of the papers/books that are often quoted in ...
2
votes
1answer
157 views

Stationary distribution for time-inhomogeneous Markov process

I have a two state, discrete time, time-inhomogeneous Markov process with transition matrix defined by $$T_i=\begin{pmatrix} 1-p_i\alpha & p_i\alpha \\ p_i\beta& 1-p_i\beta \end{pmatrix}$$ ...
0
votes
0answers
38 views

The effect of a single Markov transition on fidelity

Let $p$ and $q$ be two probability vectors of length $n$. The fidelity (or Bhattacharyya coefficient) of $p$ and $q$, is $$ F(p,q) \ := \ \sum_{i=1}^n \sqrt{p_i \cdot q_i}. $$ Let $A$ be a ...
0
votes
0answers
43 views

Ergodicity property for continuous-time Harris positive Markov process

I have posted this question on there, but got no answer. The following theorem is Theorem 13.3.3 of Meyn and Tweedie's Markov Chains and Stochastic Stability on page 328: Theorem 13.3.3. If ...
0
votes
1answer
346 views

Generating independent random variable from two correlated random variables

Suppose two random variables $X$ and $V$ are given. I am wondering what kind of condition we need to impose on joint distribution of $V$ and $X$ to make sure that there exists a random variable $Z$ ...
2
votes
0answers
66 views

Customers and Anti-Customer Queueing Problem: What is the Customer delete probability

Hello may I ask for your help? First the setting: I have got a problem with some queueing theory. The whole problem would be a grid of nodes, all nodes have an operation intensity $\mu_{i,j}$. ...
1
vote
1answer
125 views

Row-stochasticity of the Jacobian matrix of a stationary distribution

Let $P_{\mathbf{p}}$ be a $n \times n$ row-stochastic matrix whose entries are a function of a probability vector $\mathbf{p} \in \mathbf{R}_{> 0}^n$, $\sum_i p_i = 1$ and define the following ...
0
votes
0answers
71 views

Markov chains on a polyhedron

A modification of a question from Gerard Letac (1976): A m-sided q-adjacent-faced polyhedron has one of its faces "up." Each round, the polyhedron rolls so that any of the adjacent faces is now up. ...
1
vote
0answers
43 views

Oscillating Markovprocess Transition Probabilities

Suppose we have an irreducible positive-recurrent Markov process $\{X(t), t\geq0\}$ with generator $G$. Let $P(t)$ be its transition probability matrix and $\pi$ its stationary distribution. Then we ...
1
vote
0answers
56 views

Link Between Birkhoff Ergodic Theorem and Strong LLN for Harris Recurrent Markov chain

Is it possible to derive strong law of large numbers for a Harris recurrent stationary Markov chain form Birkhoff Ergodic Theorem? As I know that there is a link between SLLN for iid sample and ...
0
votes
0answers
49 views

Strong Markov Property of the joint process $(B_t,L_t)_{t\ge 0}$

Let $B=(B_t)_{t\ge 0}$ be a Brownian motion and $L=(L_t)_{t\ge 0}$ be its local time in zero. Given two strictly increasing functions $\phi_1$, $\phi_2: \mathbb R_+\to\mathbb R$ such that ...
5
votes
1answer
134 views

Relative vulnerabilities in SIS epidemic model

Consider the SIS model of epidemic spreading. There is a finite graph $G(V,E)$, link infection rates $\lambda_{ij}$ and node recovery rates $\mu_i$. There are a few initial nodes which are infected at ...
2
votes
0answers
51 views

Mappings between adaptive networks and Markov processes

Are there any known mappings between adaptive networks models (i.e. graph model representations of networks where the internal vertex dynamics and connectivity topology can change subject to specific ...
1
vote
1answer
144 views

Variation of Markov Chain Convergence Theorem

Assume the chain $\{X_n\}_{n\in\mathbb{N}}$ on the statespace $(S,\mathcal{F})$ (we may assume it is countable) is aperiodic, irreducible and positive recurrent. We denote with $\pi$ its (unique) ...
4
votes
3answers
351 views

Invariant measure of Euler-Maruyama Discretisation of an Ito diffusion

Let $(X_t)_{t \geq 0}$ be a diffusion process with dynamics governed by the stochastic differential equation \begin{equation} dX_t = b(X_t)dt + \sigma(X_t)dW_t, ~~ X_0 = x_0, \end{equation} where ...
1
vote
0answers
78 views

How to fit a stochastic matrix to given data.?

Given a data sequence of noisy observations of a 3-state Markov chain $X$ -- $y_1$,$y_2$,...$y_n$, with two transition matrices $A_1$ and $A_2$ corresponding to different regions (**) in the (unit) ...
1
vote
0answers
115 views

Spectral radius of a column stochastic matrix perturbed by a rank-1 matrix

$P\in \mathbb{R}^{n\times n}$ is an irreducible column stochastic matrix. $P$ is also diagonally dominant. $w \in \mathbb{R}^{n} $ is a strictly positive vector satisfying $w^T \mathbf{1} = 1$ where ...
1
vote
1answer
623 views

Ergodicity of a Markov chain

Hi, I'd appreciate some help on a Markov chain result I'm trying to show. I believe the following is sufficient for a continuous time Markov chain $(X_t)$ with a countable state space to be ergodic: ...