All Questions
Tagged with pr.probability markov-chains
362 questions
3
votes
3
answers
2k
views
Statistics of a simple Markov chain
Imagine a two-state Markov chain which hops between the states $\pm 1$ with probability $p<1/2$, so that the autocorrelation function after $k$ steps is
$\rho_k = (2p-1)^k$
If I take an ...
- 2,135
4
votes
1
answer
782
views
A simple problem in markov chains
I'm trying to understand a 1954 paper of Kubo intitled "Note on the stochastic theory of resonance absorption". The specific problem can be stated mathematically as follows: let $X(t)$ be a random ...
- 5,721
2
votes
1
answer
186
views
scalar diffusions are reversible
It is well known that under mild assumptions a scalar diffusion $dX_t = a(X_t) dt + \sigma(X_t) dW_t$ with invariant probability distribution $\pi$ is reversible. This is indeed not true for ...
- 17k
5
votes
2
answers
1k
views
Expectation of first positive value in random walk
Let $p$ be a parameter in $]0,1[$. Let $(X_k)_{k\geq 0}$ be an independent, identically distributed sequence of random variables, such that each $X_k$ takes values only in
$\lbrace -1, \frac{1-p}{p} \...
- 5,721
4
votes
1
answer
383
views
initial condition of a diffusion approximation
I am trying to prove that a certain sequence of Markov chains $x^N_k$ converges towards a diffusion process. The invariant measure of $x^N$ is $\pi^N$ and the Markov chain $x^N$ is started in ...
- 15.4k
1
vote
2
answers
661
views
Markov chain convergence problem.
Consider a markov chain matrix P of size n x n (n states).
P is known to be:
1- there are at least two absorbent states. one of them is denoted by null. (thus, we have that P_null,null = 1)
2- For ...
CommunityBot
- 1
5
votes
2
answers
1k
views
Is there a way to analytically compute the recurrence time of a finite Markov process?
Let $X_t$ be an ergodic (time-homogeneous) Markov process (in discrete or continuous time) on a finite state space $\{1,\dots,n\}$. Let $T(X_0)$ be the stopping time given by the infimum of times such ...
CommunityBot
- 1
1
vote
1
answer
648
views
Lower bound on the convergence rate of a specific Markov chain
I have a Markov chain $\mathbf{A} = (A_0, A_1, \ldots)$ with state space $\{0, \ldots, n\}$ which converges towards a stationary distribution $\pi$. There are a lot of well-known results on upper-...
- 15.4k
1
vote
0
answers
299
views
Markov Chain Patterns
Hi
I would like to detect repetitive patterns and deviations from these repetitions. I have historical data and can calculate probabilities for the transitions between my many states. I have ...
- 11
2
votes
1
answer
543
views
"Induced" arrivals in an M/M/1 queue?
I'm a newcomer to the realm of queueing theory, so please bear with me :)
I'd like to model web server traffic with a modified M/M/1 queue.
In the simple case we have two parameters - $\lambda$ for ...
- 14.4k
0
votes
1
answer
347
views
Where can I learn about master equation?
I am reading a paper by Dorogovstev on structure of growing complex networks with preferential linking. I need to learn master equation for this.
I need a reference for the same.
- 13.6k
4
votes
1
answer
363
views
Efficiently sampling points from an integer lattice.
Let $\mathcal{L}$ = {x$\in$ $N^n$ : ||x||$_1$ $\leq$ m} denote the set of integer points in the positive orthant of the $\ell_1$ ball of radius $m$, where $m < n$. For each $x \in \mathcal{L}$, let ...
- 1,618