The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
32 views

Validating a probability density distribution forecast model for a Markov process

Let's say we have a Markov process $X_t$, and we come up with a forecast model that takes some information from outside world and says: "value $X_{t+1}$ has probability density distribution $P_t(x)$". ...
0
votes
0answers
17 views

Gibbs sampling for Hidden Markov Model

I want to understand how to derive the update formula for Gibbs sampling for Hidden Markov Model, for example, in here: $p(z_t | \mathbf{x}, \mathbf{z}_{\setminus t}, \boldsymbol{\alpha}, ...
-1
votes
0answers
37 views

Markov operator with unique invariant measure

If a Markov operator has unique nontrival invariant measure in $L_{1}$ norm, and then I constructing a iteration series from this Markov operator and obtain a function series. Can I assert that this ...
2
votes
0answers
38 views

Variance of a functional of transition probabilities using spectral gap of Markov chain

Consider an irreducible, aperiodic, time-reversible, discrete-time Markov chain on a finite state space $S$ whose Markov kernel is $K$ and unique stationary distribution is $\pi.$ Then, reversibility ...
0
votes
0answers
20 views

Markov chain matching local time

Let $\left(X_{t}\right)_{t\geq0}$ be a Markov process taking values in a finite state space $E$. Its local time at $y\in E$ started at $x\in E$ is defined as $$ ...
0
votes
0answers
60 views

The spring Markov chain on $\mathbb{N}$

I'm trying to understand and learn more about "almost surely bounded" Markov chains on countable state spaces. I'm looking for references where I can learn how to work with more complicated examples ...
0
votes
0answers
30 views

The effect of a single Markov transition on fidelity

Let $p$ and $q$ be two probability vectors of length $n$. The fidelity (or Bhattacharyya coefficient) of $p$ and $q$, is $$ F(p,q) \ := \ \sum_{i=1}^n \sqrt{p_i \cdot q_i}. $$ Let $A$ be a ...
0
votes
0answers
34 views

Ergodicity property for continuous-time Harris positive Markov process

I have posted this question on there, but got no answer. The following theorem is Theorem 13.3.3 of Meyn and Tweedie's Markov Chains and Stochastic Stability on page 328: Theorem 13.3.3. If ...
2
votes
0answers
62 views

Customers and Anti-Customer Queueing Problem: What is the Customer delete probability

Hello may I ask for your help? First the setting: I have got a problem with some queueing theory. The whole problem would be a grid of nodes, all nodes have an operation intensity $\mu_{i,j}$. ...
2
votes
1answer
96 views

Stationary distribution for time-inhomogeneous Markov process

I have a two state, discrete time, time-inhomogeneous Markov process with transition matrix defined by $$T_i=\begin{pmatrix} 1-p_i\alpha & p_i\alpha \\ p_i\beta& 1-p_i\beta \end{pmatrix}$$ ...
0
votes
0answers
20 views

Probability distribution of a Discrete Time Markov Chain between specified start/end states and times?

You've got a DTMC $X(t)$, with state space $0\leq x \leq M$, and transition matrix $Q$. It has absorbing states $0$ and $M$ and otherwise can transition freely (no entries in $Q$ are $0$ other than ...
1
vote
1answer
119 views

Row-stochasticity of the Jacobian matrix of a stationary distribution

Let $P_{\mathbf{p}}$ be a $n \times n$ row-stochastic matrix whose entries are a function of a probability vector $\mathbf{p} \in \mathbf{R}_{> 0}^n$, $\sum_i p_i = 1$ and define the following ...
0
votes
0answers
61 views

Markov chains on a polyhedron

A modification of a question from Gerard Letac (1976): A m-sided q-adjacent-faced polyhedron has one of its faces "up." Each round, the polyhedron rolls so that any of the adjacent faces is now up. ...
1
vote
0answers
39 views

Oscillating Markovprocess Transition Probabilities

Suppose we have an irreducible positive-recurrent Markov process $\{X(t), t\geq0\}$ with generator $G$. Let $P(t)$ be its transition probability matrix and $\pi$ its stationary distribution. Then we ...
1
vote
0answers
37 views

Link Between Birkhoff Ergodic Theorem and Strong LLN for Harris Recurrent Markov chain

Is it possible to derive strong law of large numbers for a Harris recurrent stationary Markov chain form Birkhoff Ergodic Theorem? As I know that there is a link between SLLN for iid sample and ...
0
votes
0answers
44 views

Strong Markov Property of the joint process $(B_t,L_t)_{t\ge 0}$

Let $B=(B_t)_{t\ge 0}$ be a Brownian motion and $L=(L_t)_{t\ge 0}$ be its local time in zero. Given two strictly increasing functions $\phi_1$, $\phi_2: \mathbb R_+\to\mathbb R$ such that ...
2
votes
0answers
44 views

Mappings between adaptive networks and Markov processes

Are there any known mappings between adaptive networks models (i.e. graph model representations of networks where the internal vertex dynamics and connectivity topology can change subject to specific ...
30
votes
6answers
2k views

Deep Learning / Deep neural nets for mathematician

I am interested in finding out the math ideas behind the technologies that are under the umbrella of "Deep Learning" or "Deep neural nets". Most of the papers/books that are often quoted in ...
1
vote
1answer
119 views

Variation of Markov Chain Convergence Theorem

Assume the chain $\{X_n\}_{n\in\mathbb{N}}$ on the statespace $(S,\mathcal{F})$ (we may assume it is countable) is aperiodic, irreducible and positive recurrent. We denote with $\pi$ its (unique) ...
1
vote
0answers
73 views

How to fit a stochastic matrix to given data.?

Given a data sequence of noisy observations of a 3-state Markov chain $X$ -- $y_1$,$y_2$,...$y_n$, with two transition matrices $A_1$ and $A_2$ corresponding to different regions (**) in the (unit) ...
1
vote
0answers
77 views

Spectral radius of a column stochastic matrix perturbed by a rank-1 matrix

$P\in \mathbb{R}^{n\times n}$ is an irreducible column stochastic matrix. $P$ is also diagonally dominant. $w \in \mathbb{R}^{n} $ is a strictly positive vector satisfying $w^T \mathbf{1} = 1$ where ...
0
votes
0answers
37 views

Reference request for specific POMDP examples

Following is strictly for discrete-time discrete-space Markov chain. Consider a partially observed Markov decision process (POMDP) $P = \{X,O,A,P,B_a\}$. Here $X = \{x_1, \cdots, x_n\}$ refers to ...
1
vote
0answers
58 views

Theorems on stochastic Lyapunov function

Let $X_n$ be a sequence of random variables such that $$P(X_{n+1} \in A|X_m,x_m,m\leq n)= \int_A p(dw|X_n,x_n)$$ It is called a controlled Markov process. Now, suppose there exist $\epsilon_0$, ...
1
vote
0answers
134 views

Is the stationary distribution of this Markov chain uniform?

First, a little bit of background: Since 2012, Canada has decided to phase out the penny for its coinage system. Product prices may still use arbitrary cents, especially since prices do not typically ...
1
vote
0answers
27 views

Effects of merging states on the limiting distribution of a Markov Chain

Consider a discrete time, homogeneous, finite state Markov chain given by a stochastic $n\times n$ matrix $M$. We also have a cost vector $w$ of size $n$ with non-negative integer costs. The cost of ...
2
votes
1answer
181 views

Deterministic finite-state automaton driven by a Markov chain

I've stumbled on some problem, and I have the feeling that this is closed to something well-studied in dynamical systems. The problem is the following. Consider a finite-state automaton with state ...
5
votes
1answer
377 views

Can ergodic theory help to prove ergodicity of general Markov chain?

I am a beginner in ergodic theory. I have read some lecture notes(such as this and this) about it in hope that I could find something which helps to prove the ergodicity of some Markov chain taking ...
3
votes
0answers
116 views

Worst-Case Solution to (Stochastic) Matrix Inequality

EDIT: Some specific conjectures added. This problem comes with an associated stochastic process, but I phrase everything as linear algebra in case somebody from a non-probability community has seen ...
4
votes
0answers
127 views

minimal polynomial for a graph

I wonder if there is any result relating the degree $d$ of the minimal polynomial of a directed finite graph to any of its topological features - such as its diameter, or any other similar 'natural' ...
1
vote
1answer
127 views

Mixing time of a continuous time Markov chain with arbitrary rate matrix

I would like to calculate the mixing time of a continuous time starting from the rate matrix and not necessarily assuming that the time in between jumps have rate 1 - all I have is the (finite ...
0
votes
1answer
168 views

On the inverse problem of Dobrushin

Dobrushin, in this paper, looked into the following problem. Suppose We are given a Markov kernel (conditional distribution) $P_{Y|X}$. Information theorist usually call $W$ a channel. It is known ...
0
votes
0answers
23 views

Is there effective algorithm for finding “minimal discovery time” for large graphs?

Consider a large, probably sparse graph with Markovian random walkers on it. Define discovery time as time to first reach a vertex by random walk from uniform start. Are there effective ways to find ...
0
votes
1answer
102 views

Does a irreducible set of states necessarily need to be closed in a Markov chain?

I have come across two different definitions for a 'irreducible set of states' of a Markov chain. Definition 1: A subset of states $A$ of a Markov chain is irreducible if it is possible to access ...
1
vote
0answers
86 views

Conditional probabilities in epidemic model

I was contemplating an epidemic model where infection and recovery rates are determined by links. Here node $i$ is infected first and recovers at a rate $\mu_i$. For all other nodes, the recovery is ...
2
votes
1answer
208 views

Stationary distribution of Markov chain

Suppose I have a discrete time Markov chain $\boldsymbol{X}$ with state space $\mathbb{R}^+$. The chain is $\psi$-irreducible, aperiodic, atomless and has an invariant measure $\pi$. If $\pi$ is ...
1
vote
1answer
171 views

Can ergodic theorem be used here [closed]

Suppose I have an ergodic Markov Chain $\{X_n\}$ where $X_n$ are bounded. Now, Can I say anything on the limit $$ \lim_{n\to\infty} \frac{1}{n}\ln E\left[e^{\sum_{i=0}^{n} X_i}\right]$$ I don't ...
1
vote
1answer
197 views

Double Markovity

Suppose we have a double Markov relation for three random variables $X$, $Y$ and $W$ as follows $$X\to W\to Y,$$ and $$X\to Y\to W.$$ How to prove that there exist functions $f$ and $g$ such that ...
0
votes
1answer
67 views

Constructing a transition matrix of a time-homogeneous, finite Markov chain with full support stationary distribution

is there a way to construct a transition matrix of a time-homogeneous, finite Markov chain such that the stationary distribution always has full support (this is equivalent to all states of the chain ...
0
votes
1answer
48 views

DTMC random walk model [closed]

For a discrete Markov chain random walk with p < 0.5 with state space S= {0,1,2..} What is the stationary distribution? I could use any help. Thank you
5
votes
0answers
155 views

Maximal inequalities for square of partial sums

Let $S_n = \sum_{i \leq n} X_i$ be the partial sums of a nice sequence of random variables $X_i$. In my application, $X_i$ is a functional of a finite-state, irreducible, aperiodic Markov chain, so ...
1
vote
0answers
67 views

Simultaneous multiple perturbations in Markov chain Monte Carlo

I'm coding a McMC algorithm for geophysical applications. Using the Metropolis-Hastings scheme to accept/reject the proposed models is smth that i thought i completely understood, but i don't. To be ...
3
votes
1answer
160 views

Approximating Markov chains by Brownian motion

I would like a result along the following lines to be true, but haven't been able to locate it in the literature; pointers would be welcome. Let $X_t$ be a finite-state, irreducible, aperiodic Markov ...
1
vote
1answer
108 views

N random walkers that hit node v in a graph

Consider a finite, undirected graph G, with uniform edge weights. Assume that there are n number of random walkers that will start at different nodes (lets say n=3, hence the random walkers will start ...
1
vote
0answers
71 views

Nonstationary Markov chain maximal inequality

Let $X_i$ be a (finite-state, irreducible, aperiodic) Markov chain, not necessarily stationary. (That is, it doesn't start from the invariant distribution; I'm happy to have it be time-homogeneous if ...
1
vote
0answers
331 views

random walk with reflecting barriers [closed]

Consider a random walk on the line 1,...,d. You start at point 1. At each step you flip a coin: heads means go left, tails means go right. If you're at 1 and get a heads, just stay where you are (same ...
1
vote
1answer
156 views

Gibbs sampler with linear constraints

My problem concerns the estimation of truncated multivariate normal distributions under constraints. Let $X_1$ and $X_2$ two random variables following normal distributions ...
4
votes
3answers
656 views

How to explain “Feller process” to an undergraduate student?

I had to explain in informal terms what a Feller process was, to undergraduate students who understand Markov property, Poisson processes and such. It was easy to define Levy process as generalisation ...
7
votes
1answer
217 views

First Collision Time for k Random Walkers on a Torus

I consider $k$ random walkers on $\mathbb{Z}^{d}/n \mathbb{Z}^{d}$, the $d$-dimensional torus of side length $n$. More precisely, I will define a Markov chain $Z_{t} = (X_{t}[1], \ldots, X_{t}[k])$ ...
2
votes
0answers
40 views

Markov decision processes: action set revealed at point of decision

I have a problem which looks like a finite horizon Markov decision process (MDP), except the action space at each time is revealed at the decision making point. There is no way to know before hand the ...
6
votes
2answers
554 views

References for a physicist migrating to stochastic processes

I've studied "Markov Chains" - Norris and "Measure, Integral and Probability" - Capinski, Kopp. Now, I'm looking for a couple of books (or other references) that help me bridging these two topics. ...