All Questions
Tagged with pr.probability markov-chains
362 questions
6
votes
1
answer
387
views
Quasi-stationary distribution for a death process
In the paper, Survival in a quasi-death process by van Doorn and Pollett, the quasi-stationary distribution of a transient CTMC is discussed and QSD for a simple death process is derived.
Consider a ...
CommunityBot
- 1
8
votes
1
answer
174
views
Equalizing Geometric means of Graph Cycles
Consider a strongly connected directed graph $G$. I have been stuck on the following question: can you assign real numbers in $[0,1]$ to each edge of $G$ so that the geometric mean of all cycles are ...
CommunityBot
- 1
1
vote
1
answer
242
views
Two types of random walkers on square lattice
Consider a two dimensional square lattice ($n$ by $n$), which is our space $S$ (each point labelled by an index $1\to n^2$), containing two types of particles, distinguished here by either an index $1$...
CommunityBot
- 1
3
votes
0
answers
144
views
The spring Markov chain on $\mathbb{N}$
I'm trying to understand and learn more about "almost surely bounded" Markov chains on countable state spaces. I'm looking for references where I can learn how to work with more complicated examples ...
- 19.6k
1
vote
0
answers
46
views
Is there an effective algorithm for finding "minimal discovery times" for large graphs?
Consider a large, probably sparse graph with Markovian random walkers on it.
Define the discovery time as the expected time to first reach a vertex by
random walk from a uniform start. Are there ...
- 19.6k
7
votes
1
answer
296
views
Existence of Limiting Distribution for Moving Regions in Stat. Phys. Models
As the title (hopefully) suggests, I've been trying to prove (or disprove) the existence of a limiting distribution for a certain projection in a statistical physics model. I'll give the details of ...
- 188k
1
vote
0
answers
44
views
Validating a probability density distribution forecast model for a Markov process
Let's say we have a Markov process $X_t$, and we come up with a forecast model that takes some information from outside world and says: "value $X_{t+1}$ has probability density distribution $P_t(x)$". ...
- 3,085
2
votes
1
answer
2k
views
Stationary distribution for time-inhomogeneous Markov process
I have a two state, discrete time, time-inhomogeneous Markov process with transition matrix defined by
$$T_i=\begin{pmatrix}
1-p_i\alpha & p_i\alpha \\
p_i\beta& 1-p_i\beta
\end{pmatrix}$$
...
- 544
0
votes
0
answers
117
views
Ergodicity property for continuous-time Harris positive Markov process
I have posted this question on there, but got no answer.
The following theorem is Theorem 13.3.3 of Meyn and Tweedie's Markov Chains and Stochastic Stability on page 328:
Theorem 13.3.3. If $\Phi$ ...
CommunityBot
- 1
0
votes
1
answer
408
views
Generating independent random variable from two correlated random variables
Suppose two random variables $X$ and $V$ are given. I am wondering what kind of condition we need to impose on joint distribution of $V$ and $X$ to make sure that there exists a random variable $Z$ ...
CommunityBot
- 1
1
vote
1
answer
218
views
Row-stochasticity of the Jacobian matrix of a stationary distribution
Let $P_{\mathbf{p}}$ be a $n \times n$ row-stochastic matrix whose entries are a function of a probability vector $\mathbf{p} \in \mathbf{R}_{> 0}^n$, $\sum_i p_i = 1$ and define the following ...
- 128k
1
vote
1
answer
363
views
Variation of Markov Chain Convergence Theorem
Assume the chain $\{X_n\}_{n\in\mathbb{N}}$ on the statespace $(S,\mathcal{F})$ (we may assume it is countable) is aperiodic, irreducible and positive recurrent. We denote with $\pi$ its (unique) ...
- 17k
1
vote
0
answers
101
views
How to fit a stochastic matrix to given data.?
Given a data sequence of noisy observations of a 3-state Markov chain $X$ -- $y_1$,$y_2$,...$y_n$, with two transition matrices $A_1$ and $A_2$ corresponding to different regions (**) in the (unit) ...
3
votes
1
answer
1k
views
Ergodicity of a Markov chain
Hi,
I'd appreciate some help on a Markov chain result I'm trying to show. I believe the following is sufficient for a continuous time Markov chain $(X_t)$ with a countable state space to be ergodic:
...
- 200
2
votes
0
answers
970
views
Is the stationary distribution of this Markov chain uniform?
First, a little bit of background: Since 2012, Canada has decided to phase out the penny for its coinage system. Product prices may still use arbitrary cents, especially since prices do not typically ...
- 3,587
3
votes
0
answers
158
views
Worst-Case Solution to (Stochastic) Matrix Inequality
EDIT: Some specific conjectures added.
This problem comes with an associated stochastic process, but I phrase everything as linear algebra in case somebody from a non-probability community has seen ...
- 31
2
votes
1
answer
903
views
Mixing time of a continuous time Markov chain with arbitrary rate matrix
I would like to calculate the mixing time of a continuous time starting from the rate matrix and not necessarily assuming that the time in between jumps have rate 1 - all I have is the (finite ...
- 229
0
votes
1
answer
207
views
On the inverse problem of Dobrushin
Dobrushin, in this paper, looked into the following problem. Suppose We are given a Markov kernel (conditional distribution) $P_{Y|X}$. Information theorist usually call $W$ a channel. It is known ...
CommunityBot
- 1
6
votes
2
answers
912
views
References for a physicist migrating to stochastic processes
I've studied "Markov Chains" - Norris and "Measure, Integral and Probability" - Capinski, Kopp. Now, I'm looking for a couple of books (or other references) that help me bridging these two topics. ...
CommunityBot
- 1
4
votes
0
answers
153
views
Mixing time for dimers on the square-octagon graph
Consider the "fortress graph" of order $n$ (see Figure 9 of http://faculty.uml.edu/jpropp/tiling/www/mdblum/arctic.html). It's been known empirically for twenty years that if one turns the set of ...
- 1,872
1
vote
0
answers
111
views
Conditional probabilities in epidemic model
I was contemplating an epidemic model where infection and recovery rates are determined by links. Here node $i$ is infected first and recovers at a rate $\mu_i$. For all other nodes, the recovery is ...
- 519
1
vote
1
answer
352
views
Double Markovity
Suppose we have a double Markov relation for three random variables $X$, $Y$ and $W$ as follows
$$X\to W\to Y,$$ and $$X\to Y\to W.$$
How to prove that there exist functions $f$ and $g$ such that
$$...
- 303
6
votes
0
answers
337
views
Maximal inequalities for square of partial sums
Let $S_n = \sum_{i \leq n} X_i$ be the partial sums of a nice sequence of random variables $X_i$. In my application, $X_i$ is a functional of a finite-state, irreducible, aperiodic Markov chain, so ...
- 1,069
1
vote
0
answers
92
views
Simultaneous multiple perturbations in Markov chain Monte Carlo
I'm coding a McMC algorithm for geophysical applications.
Using the Metropolis-Hastings scheme to accept/reject the proposed models is smth that i thought i completely understood, but i don't. To be ...
- 11
12
votes
3
answers
4k
views
How to explain "Feller process" to an undergraduate student?
I had to explain in informal terms what a Feller process was, to undergraduate students who understand Markov property, Poisson processes and such. It was easy to define Levy process as generalisation ...
- 1,769
1
vote
0
answers
181
views
Nonstationary Markov chain maximal inequality
Let $X_i$ be a (finite-state, irreducible, aperiodic) Markov chain, not necessarily stationary. (That is, it doesn't start from the invariant distribution; I'm happy to have it be time-homogeneous if ...
- 1,069
7
votes
1
answer
337
views
First Collision Time for k Random Walkers on a Torus
I consider $k$ random walkers on $\mathbb{Z}^{d}/n \mathbb{Z}^{d}$, the $d$-dimensional torus of side length $n$. More precisely, I will define a Markov chain $Z_{t} = (X_{t}[1], \ldots, X_{t}[k])$ ...
- 1,411
2
votes
1
answer
665
views
Transition probabilities in coupled Markov chains
I know that for a continuous-time Markov chain, the probability of transition from time $0$ to $t$ is given by $P(t)=e^{Q(t)t}$. I have a system of $N$ interdependent continuous-time Markov chains ...
- 1,769
3
votes
1
answer
403
views
convergence rate of occupation measure of ergodic Markov Chain
Given an ergodic Markov chain $(X_n)_{n\geq 1}$ in $R^d$with $\pi$ as the invariant distribution of the transition kernel, under good conditions we have that the empirical occupation measure converges ...
- 6,541
3
votes
0
answers
305
views
Nonlinear Markov process
Consider the following nonlinear $\mathbb{R}$-valued stochastic recursive sequence:
$ X_{n+1} = F(X_n) + W_{n+1}, \quad (W_n)_{n\ge1} \stackrel{ \scriptsize \mathrm{i.i.d.} }{ \sim } \phi. $
How can ...
- 61
2
votes
0
answers
199
views
CLT for a Markov Renewal Process
Suppose $(X,T)=\{(X_n,T_n)\}_{n\geq0}$ is a Markov renewal process, where $X$ is a finite-state, discrete-time Markov chain with state space $\{1,2,...,R\}$. $T$ is the additive component, more ...
- 41
6
votes
1
answer
218
views
Finding cohesive (low exit probability) sets in a Markov process
The following is a fact about Markov chains that came up in a game theory paper. The purpose of this question is to ask if related notions or similar results are found elsewhere in probability, or are ...
- 1,068
4
votes
1
answer
207
views
Log-likelihood in regime switching
Not sure if this question is too simple to be asked here...
In the following paper
Cho, Jin Seo, and Halbert White. "Testing for regime switching." Econometrica 75.6 (2007): 1671-1720. doi: 10.1111/...
- 215
1
vote
1
answer
149
views
Minimal variance for phase-type distributions?
Let $\mathcal{D}(m)$ be the set of phase-type distributions constructed from $m+1$-state Markov chains. Recall that the coefficient of variation of a distribution $D$ is the ratio of the standard ...
- 8,501
3
votes
0
answers
494
views
Maximization of a total variation distance subject to another total variation distance in Markov chain
Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the ...
- 1,109
2
votes
0
answers
162
views
An optimization in Markov Chain
We are given two correlated random variables $V$ and $X$ supported over a finite alphabets $\mathcal{V}$ and $\mathcal{X}$. Suppose the marginal $P_V$ and conditional distribution $P_{X|V}$ are ...
0
votes
1
answer
2k
views
Stationary distribution in general Markov Chains
This is just a reference request for a result which is very general, useful and should be well-known, but I've failed to find a good reference to cite.
The problem is to define the "most natural" ...
CommunityBot
- 1
1
vote
1
answer
160
views
Optimum control of a probabilistic automaton
Suppose we have a probabilistic automaton and we assign a weight to each state. An "interaction strategy" would be a fixed map from states to inputs. Any interaction strategy could be used to ...
CommunityBot
- 1
3
votes
4
answers
681
views
Does the variance of a continuous time, time homogeneous, Markov process starting from one point necessarily not decrease?
Let $x_t$ be a zero mean, time homogeneous Markovian process (chiefly look at the case where the value is in $1$ dimension) over time $t$ starting from $x_0=0$. Is it necessary that, in continuous ...
- 10.3k
3
votes
1
answer
2k
views
Hitting time probability in a Random Walk with possibility to die.
A Random Walker can move of one unit to the right with probability $p$, to the left with probability $q$ and it can jump again to the starting point with probability $r$ and die. Naturally $p+q+r=1$. ...
- 2,239
1
vote
0
answers
196
views
The problem of the drunkard in a valley [closed]
We consider a Markov chain on a subset of positive integers S = {0, 1, 2, 3, .......N}, with transition probabilities defined as follows:
The chain jumps only one unit to the left or right.
p(i, j) =...
2
votes
1
answer
356
views
The first eigenvalue of a branching process matrix
Let $M$ be the real square matrix of a typed branching process, such that $M_{ij}$ is the expected value of offspring of type $j$ emanating from type $i$.
We know that if the first eigenvalue if $M$ ...
CommunityBot
- 1
7
votes
2
answers
698
views
Convergence rate of the convolution of almost uniform measures on $\mathbb{Z}_p$
Statement Given a finite abelian group $G$ and two independent random variables $X,Y$ taking values in $G$ and satisfying $d_{TV}(X,U_G)\leqslant \delta$ and $d_{TV}(Y,U_G)\leqslant \delta$ (where $...
CommunityBot
- 1
5
votes
2
answers
516
views
Anticoncentration of the convolution of two characteristic functions
Edit: This is a question related to my other post, stated in a much more concrete way I think.
I am interested in anything (ideas, references) related to the following problem:
Suppose that $A \...
- 696
4
votes
0
answers
264
views
Generalized Markov Processes on CW complexes of dimension > 1
Markov processes have a large variety of applications to physics and chemistry (as well as many other fields). Such processes are formulated on graphs, i.e., CW complexes of dimension one. It is ...
CommunityBot
- 1
3
votes
0
answers
95
views
Best convergence rate for convolutions on $\mathbb{Z}_p$
Suppose, that we have sequence of i.i.d variables $X_1,\ldots,X_n$ taking values in $\mathbb{Z}_p$ such that $d_{TV}(X_1,U) < \delta$.
How fast, in terms of $\delta$ and $n$ does the sum $X_1+\...
- 696
4
votes
3
answers
4k
views
Stationary distribution for bipartite graph
I was wondering if there is any stationary distribution for bipartite graph? Can we apply random walks on bipartite graph? since we know the stationary distribution can be found from Markov chain, but ...
- 1,037
3
votes
0
answers
770
views
Kullback-Leibler Divergence of Stationary Distributions of Markov chains
Consider two finite Markov chains on the same state space, both assumed to be irreducible, with transition matrices $P$ and $Q$ and associated stationary distributions $\pi$ and $\tilde \pi$. Is it ...
- 418
3
votes
1
answer
188
views
Markov Chains based on sampled transition probabilities [closed]
If I have a process that transitions between states with some set, unknown probability, I can sample to find the transition probability. This probability is a sample average, with a well understood ...
- 1,902
0
votes
1
answer
2k
views
Markov Chain: state reduction
Hi I am trying to understand a proof in a paper (written by Isaac Sonin), I don't know if anyone could give me a clarification on the following:
Firstly we have a Markov chain $\{Y_k\}$ with finite ...
- 23