# Tagged Questions

The tag has no usage guidance.

108 views

### Hierarchical (Recursive) Random Walk Model

Consider the following hierarchical (recursive) random walk model. For the first level $\ell=1$, let $\{X_t^{(\ell)}\}_{t=1}^{T}$ be a (discrete) random walk. For the next level $\ell=2$, we ...
620 views

### Coin Toss Probabilities like Penney's Game

Generate a binary number, using coin toss. Until you receive a predefined terminating sequence. What is the probability that the number is a multiple of some $k$. For example, the terminating ...
69 views

### Is there an easy way to convert a non-deterministic optimal policy to a deterministic optimal policy for a given MDP?

For a MDP (Markov Decision Process) is there an easy way to convert a non-deterministic optimal policy into a deterministic optimal policy? The trivial way will take $O(|\mathcal{A}|^{|\mathcal{S}|}$...
33 views

### Convergence of an inhomogeneous markov chain

A markov chain is defined as $X_t=F(X_{t-1})X_{t-1}$, where $X_t$ and $X_{t-1}$ are both vector. So the transition matrix depends on the current states. I want to show that for any given initial ...
283 views

### How to sample a uniform random polyomino?

A polyomino is formed by joining finitely many unit squares edge to edge. It may be regarded as a finite subset of the regular square tiling with a connected interior. In particular, for us, ...
22 views

### markov chains basic property proof [closed]

why P(Xn+1=j,Xn=i) is equal to P (Xn=i) in the proof? proof img
315 views

### Deterministic finite-state automaton driven by a Markov chain

I've stumbled on some problem, and I have the feeling that this is closed to something well-studied in dynamical systems. The problem is the following. Consider a finite-state automaton with state ...
109 views

### Basic Definition and Notations in RWRE

From the definition of Zeitouni's lecture notes on RWRE: $(V, E)$ is a special graph, and $N_v:= \{k \in V: (v,k) \in E\}$ is the neighborhood of $v \in V$. $\Omega = \prod_{v \in V} M_1(N_v)$ ...
243 views

### minimal polynomial for a graph

I wonder if there is any result relating the degree $d$ of the minimal polynomial of a directed finite graph to any of its topological features - such as its diameter, or any other similar 'natural' ...
259 views

### Does random walk have more concentration surrounding the origin?

Consider a simple random walk $S_n$ on one dimension, starting at $0$. In this case, $S_n$ fluctuates between $-\infty$ and $\infty$, but intuition says that it might stay more often in an interval ...
171 views

### Frequency of visiting states in Markov chains

Given a finite, ergodic Markov $\{X_i\}$, and two natural numbers $a>b$. Let $$p=P\left[\forall n, \sum_{k=n}^{n+a-1} \mathbf{1}_m(X_k)\leq b\right]$$ where $\mathbf{1}_m(X_k) =1$ if $X_k=m$ and 0 ...
22 views

### showing that a matrix has repetitive values?

Here my primary aim is to calculate the stationary distribution of a DTMC using left-eigen values i.e, $\pi = \pi*P$. But for some matrices, I observe that some states a same stationary probability. ...
69 views

### How can I efficiently approximate the stationary distribution of an infinite CTMC with a sparse rate matrix?

I am looking for methods to approximate the stationary distribution of an infinite CTMC with a sparse rate matrix. Each row and column of the rate matrix has a finite number of non-zero elements. ...
262 views

168 views

### The necessary sufficient condition for recurrence of a Markovian random walk

Suppose $\sigma_{1},\sigma_{2},...$are i.i.d random variables.$S_{0}=0$. Define $S_{n}=S_{0}+\sum_{i=1}^{n}\sigma_{i}$, then ${S_{n}}$ is a Markovian random walk. I want to figure out the necessary ...
26 views

### Dependency of the error term on the states, in the definition of the transition rates of a continuous time Markov chain

I think this is certainly not a research or graduate level question. But I didn't get any answer from math.stackexchange.com. I'm studying G.F.Lawler's stochastic process book. There he defines the ...
194 views

### Local Markov implies global Markov

Let $G=(V,E)$ be a finite simple graph, and let $\{X_i\}_{i \in V}$ be a collection of random variables associated with the vertices of $G$. The joint distributions of these r.v.s is a Markov Random ...
7k views

### Solving a Rubik's cube via a series of randomly selected (quarter-turn) Singmaster moves

In July of 2010, Tomas Rokicki, Herbert Kociemba, Morley Davidson, and John Dethridge demonstrated (computationally) that a $3\times3\times3$ Rubik's cube, starting in an arbitrary configuration, can ...
373 views

### Why does the overhand shuffle converge to the uniform distribution on $S_n$?

Pemantle 1989 proves, among other things, that the Markov chain on $S_n$ induced by repeatedly and independently performing an overhand shuffle on a deck of $n$ cards is ergodic and has limiting ...
132 views

### markov processes and ergodic theory

For an ergodic Markov Chain $$\frac{1}{N}\sum_{i=1}^n f(X_i) \rightarrow E_\pi[f]$$ where $\pi$ is the invariant distribution. I am also dealing with a Markovian process (a state space model to ...
39 views

### Discrete time hidden markov process [closed]

I am dealing with a hidden Markov model for variable $X_{t+1}$ where $X_{t+1}$ = $\alpha_{t}$$X_{t} + (1-\alpha_{t})$$Z_{t}$ $X_{t}$ is an indicator variable indicating wether an individual is ...
637 views

### Can ergodic theory help to prove ergodicity of general Markov chain?

I am a beginner in ergodic theory. I have read some lecture notes(such as this and this) about it in hope that I could find something which helps to prove the ergodicity of some Markov chain taking ...
323 views

### Does every (generalized?) Markov chain admit transition probabilities?

To pose the question let us start by recalling the following notions: Transition Probabilities. A transition probability matrix between two measurable spaces $(S,\mathcal{S})$ and $(V,\mathcal{V})$...
130 views

### Neat definition of Harris Ergodicity

I can't find any reference where the definition of Harris Ergodicity for Continuous time Markov processes is defined. a) What would be exactly the definition? b) What reference could be helpful? ...
60 views

### Choose uniformly from fixed-length paths in $[0,n]\cap\mathbb{Z}$ with fixed start and end

Let $X_k$ be a symmetric (discrete time) random walk on $\mathbb{Z}$ and let $m,n\in\mathbb{N}$. I want to chose uniformly from the paths of $X_k$, which start at $0$ stay in $[0,n]\cap\mathbb{Z}$ ...
153 views

### Must rows of a transition matrix be distinct?

Is it true that for all continuous time Markov processes on a countable state space $S$, we have all rows of the transition matrix $\mathbf{P}_t$ are distinct for all time $t\in[0,\infty)$ ? This ...
34 views

### Strong Markov vector-valued process from component strong Markov process and independence

I want to prove that if $X$ and $Y$ are (continuous time) independent strong markov $\mathbb{R}$-valued processes w.r.t. their natural filtrations $\mathcal{F}^X_t$ and $\mathcal{F}^Y_t$, that the ...
69 views

23 views

### Properties of a map regarding the space of invariant probability measures for controlled Markov process

Let us consider a controlled Markov process with the transition kernel $p(dy|x,\theta)$ ($\theta$ being the control parameter. Now, consider the map $\theta \to I(\theta)$ where $I(\theta)$ is the ...
Consider a discrete time Markov chain $X_t$ on some finite state space $\mathcal{S}$ with transition matrix $P$. Now consider a process $Y_t$ also on $\mathcal{S}$, which conditioned on $X_{t}=s$ ...