# Tagged Questions

The tag has no usage guidance.

14 views

### Unique Stationary Distribution of A Markov Chain

I have a Markov Chain like $Y_i=\sum_n\pi_{n,i}(Y)Y_n$, i=1,2,3...N. So the Markov chain has N states and the transition matrix depends on the vector $\textbf{Y}$. I am wondering what conditions $\pi$ ...
91 views

### Asymptotic Growth of Markov Chain

I asked the following question one week ago at math.stackexchange but didn't receive a response, so I want to give it here another try: I'm interested in the following problem: We have got a time-...
188 views
+50

### Hierarchical (Recursive) Random Walk (also known as Hierarchical Hidden Markov Model)

Consider the following hierarchical (recursive) random walk model, which is also known as the hierarchical hidden Markov model in computer science (https://en.wikipedia.org/wiki/...
34 views

### Convergence of an inhomogeneous markov chain

A markov chain is defined as $X_t=F(X_{t-1})X_{t-1}$, where $X_t$ and $X_{t-1}$ are both vector. So the transition matrix depends on the current states. I want to show that for any given initial ...
292 views

### How to sample a uniform random polyomino?

A polyomino is formed by joining finitely many unit squares edge to edge. It may be regarded as a finite subset of the regular square tiling with a connected interior. In particular, for us, ...
260 views

### Does random walk have more concentration surrounding the origin?

Consider a simple random walk $S_n$ on one dimension, starting at $0$. In this case, $S_n$ fluctuates between $-\infty$ and $\infty$, but intuition says that it might stay more often in an interval ...
22 views

### showing that a matrix has repetitive values?

Here my primary aim is to calculate the stationary distribution of a DTMC using left-eigen values i.e, $\pi = \pi*P$. But for some matrices, I observe that some states a same stationary probability. ...
171 views

### Frequency of visiting states in Markov chains

Given a finite, ergodic Markov $\{X_i\}$, and two natural numbers $a>b$. Let $$p=P\left[\forall n, \sum_{k=n}^{n+a-1} \mathbf{1}_m(X_k)\leq b\right]$$ where $\mathbf{1}_m(X_k) =1$ if $X_k=m$ and 0 ...
1k views

### Random walk to stay in an interval forever

Consider a random walk on the real time, starting from $0$. But this time assume that we can decide, for each step $i$, a step size $t_i>0$ to the left or the right with equal probabilities. To ...
110 views

### Basic Definition and Notations in RWRE

From the definition of Zeitouni's lecture notes on RWRE: $(V, E)$ is a special graph, and $N_v:= \{k \in V: (v,k) \in E\}$ is the neighborhood of $v \in V$. $\Omega = \prod_{v \in V} M_1(N_v)$ ...
101 views

### Exact formula for computing n-step transition probability of random walks with self-transitions

Consider a semi-infinite random walks $X_n$, $n=0,1,2,\ldots$, whose state space is a set of consecutive integers and whose one-step transition probabilities are $P_{ij}=\mathrm{Pr}\{X_{n+1}=j|X_n=i\}$...
76 views

### Looking for an exposition of a certain theorem of Talagrand

The following is a theorem by Talagrand (as stated here, http://arxiv.org/pdf/1511.08609v1.pdf), Let $(X, \mu)$ be a probability space. Let $F : X \rightarrow \{0,1\}$ be a family of functions ...
63 views

176 views

### Mixing time of lazy random walk on the directed cycle $C_n$

Briefly: A hint (if this is easy), reference or derivation would be of great help. The question Let $C_n$ be the directed cycle with loops in each of its $n$ vertices, and consider the random walk ...
17 views

### Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models. Consider a hidden Markov model (HMM) with ...
48 views

88 views

### Finite hitting time implies hits at any finite time?

I was wondering about the following problem: Assume we have a state space $S:=\mathbb{Z}$ and a Markov chain, such that we can go from any state $x$ to some state $y$ with positive probabilities, i.e....
27 views

223 views

### Calculate the KL divergence between two transition matrices

I want to calculate how different two markov transition matrices are. For example: $\begin{pmatrix} .2 & .8 \\ .1 & .9 \end{pmatrix}$ and \$\begin{pmatrix} .3 & .7 \\ .1 & .9 \...